{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f40657e-cc55-5f02-bb56-b6cdf7bb68f9","task_key":"default--validation--48461","task_revision_id":"1","upstream_id":"48461","short_description":"HellaSwag validation 48461","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"Then a man is standing on the track stretching, he jumps, and then comes back talking to the screen before he is shown clearing another bar. after\",\"endings\":\"['he does this, he jumps again and rings out to fifth ring.', ', various images of pro athletes and high jumpers are shown before the man comes back to talk.', 'that he is shown again in another suit jumping, he dances and falls to the ground again and a replay is shown of him falling.', ', he throws the large green ball over his shoulder and runs again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}