{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2efcdc5a-bbae-5608-81b3-515dbdbce8f2","task_key":"default--validation--46135","task_revision_id":"1","upstream_id":"46135","short_description":"HellaSwag validation 46135","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A commercial displays advertising a sea wheeze event by showing groups of women beginning a marathon, live music, and live group yoga all done outdoors. a group of women\",\"endings\":\"['perform yoga and start a marathon interspersed with images of people juggling, cheer leading and smiling and having fun.', 'begins a day together together while the man runs from client to client and top to bottom to make sure the trek is smooth.', 'are standing at ice cream tables and eating ice cream.', 'are seen sitting in curved chairs from many years ago with tan skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}