{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81f4ca2a-c287-5e45-935b-f6880e666b60","task_key":"default--validation--34127","task_revision_id":"1","upstream_id":"34127","short_description":"HellaSwag validation 34127","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to eight strand braid [title] Split your strands into two, equal groups. [step] You should have four strands in the left group, and four strands in the right. Spread the strands out so that you can see all of them.\",\"endings\":\"['Repeat this step for the left strand and the right braid. Each strand should now be divided into four groups.', 'You will begin the braiding process with the two strands in the far left groups. [substeps] You should divide your strands into two groups again, joining them together at the very far right group.', 'Keep a wide space between the two groups. [title] Weave the first left strand through the other left strands.', 'Apply makeup pressure to help you place your knot, if applicable. Use a pastry brush to gently apply makeup to the underside of your strands.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}