{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ea7a86c2-8e89-549a-b0c4-e5b692c4aaa7","task_key":"default--validation--48460","task_revision_id":"1","upstream_id":"48460","short_description":"HellaSwag validation 48460","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"Another male is then shown as he takes off and clears the bar of the high jump and there's an instant replay. then a man\",\"endings\":\"['is standing on the track stretching, he jumps, and then comes back talking to the screen before he is shown clearing another bar.', \\\"next to the bars is shown white knuckles and feels a little hurt that this particular person beat him and he let's him go.\\\", \\\"takes off and he's in a facility with many onlookers as he high fives a man behind a bar.\\\", 'appears to show how fast and how hard he can do it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}