{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd16ef82-a1d0-59a5-b302-c554bd86bc8f","task_key":"default--validation--18186","task_revision_id":"1","upstream_id":"18186","short_description":"HellaSwag validation 18186","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man is bent over holding on to a weight with lots of weight on it. He squats down kind of and picks it up, it looks very heavy and kind of painful. he\",\"endings\":\"['lifts up above his head and jumps his feet apart.', 'drops it down and just sits there relaxing.', \\\"slams it against the wall again but it's not so hard this time.\\\", 'almost drops the weight and sits back up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}