{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6a76111-0bf3-5bc2-8810-bed1341e492d","task_key":"default--validation--12451","task_revision_id":"1","upstream_id":"12451","short_description":"HellaSwag validation 12451","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to thicken stew [title] Sprinkle a small amount of breadcrumbs into the stew liquid. [step] Stir to mix well. [substeps] Use dried or frozen breadcrumbs.\",\"endings\":\"['Any kind is acceptable, but dry is fine. It needs to be soft but not runny.', \\\"They are often dried or frozen, which means they won't become heavy in the stew. [title] Cook for 30 minutes or until fully cooked.\\\", '[title] Allow for a little more cooking, then check to see if the stew has thickened adequately. [step] If not, repeat with another sprinkling of breadcrumbs.', 'The starchy matter will lend them flavor. [title] Simmer the stew for 30 to 60 minutes, cooking at a medium-low heat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}