{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77355943-c1be-5b84-86f3-2280fe8ae7da","task_key":"default--validation--37572","task_revision_id":"1","upstream_id":"37572","short_description":"HellaSwag validation 37572","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A man and a woman run and exercise in a gym. The woman and the man talk in the gym and give advice if a person is cramped up. after\",\"endings\":\"[', the man goes in the house to grab a towel and a pair of gym pants and tie his shoe and join the woman and the man in gym.', ', the man puts his arms up to jump at the gee gee and performs huge leaps.', ', the man and woman talk in the courtroom.', ', the woman and the man continue exercising.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}