{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aedc8704-0653-5b11-b617-bd9443e507bc","task_key":"default--validation--34538","task_revision_id":"1","upstream_id":"34538","short_description":"HellaSwag validation 34538","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to eat ugli fruit [title] Know when and where to look. [step] Ugli fruit are only available from december through april, and you might need to go to a specialty food store to find them. [substeps] The ugli fruit is actually another name for the jamaican tangelo.\",\"endings\":\"['Tangelo fruit products are a single fruit that comes from all faiths. They are also made with a diversity of fruits, and they are available from virtually anywhere in the country.', 'It is extremely high in fructose, so if you plan to eat it, make sure to bring enough fruit. Note that this fruit does have a sweet, powdered fruit, so you should avoid adding fruit with sour parts.', 'The ugli fruit only has the year 5 fruit on it, and this fruit only has 2 leaves. The fruit also has fruit, such as pineapple, strawberries, and limes.', \\\"It was discovered in jamaica and became one of the country's exports starting in 1914. While the fruit is imported to the united states, its supply is fairly limited, so it can be somewhat expensive.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}