{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f0a80c0-2f33-5e5f-b8c4-a184b7983195","task_key":"default--validation--40320","task_revision_id":"1","upstream_id":"40320","short_description":"HellaSwag validation 40320","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"The people are shown slowly jogging in this marathon. there\",\"endings\":\"['is footage of a man hitting the pinata over and over and a soccer match is shown in slow motion.', 'is another pause during the marathon and the men are showing that they got due time.', 'is only one man who juggles white balls in his hands while jogging backwards throughout the whole thing.', 'are several viewer who have difficulty just breathing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}