{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bf9caf06-7111-5aa9-8036-ecb80f162445","task_key":"default--validation--43613","task_revision_id":"1","upstream_id":"43613","short_description":"HellaSwag validation 43613","config":"default","split":"validation","body":"{\"activity_label\":\"Blowing leaves\",\"ctx\":\"This elderly man is blowing the leaves out of the way using a leaf blower. First the camera is giving viewers a close up view only showing the man's face. then the camera\",\"endings\":\"['man is showing viewers how the elderly man placed a close up of his face using an 8 inch leaf blower.', \\\"briefly focuses on the person's hands on the rake as he continues raking and leaves fall on the ground.\\\", 'returns to the same young man blowing leaves with the blower.', \\\"zooms out and shows the man blowing the leaves using a machine and when he's done half doing his job, he turns the leaf blower off.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}