{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa62e864-f1aa-5ced-adb0-57faf98a4b29","task_key":"default--validation--32518","task_revision_id":"1","upstream_id":"32518","short_description":"HellaSwag validation 32518","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"She returns to lathering her face and waits in anticipation. While she waits, she plays some instruments and with her cat. finally\",\"endings\":\"[', she arrives and wobbles on her knees as she positions herself for the performance.', 'the man arrives, and he kisses her with surprising energy, as well as claps his hands.', ', she blows a kiss to the cat and ends.', ', she rinses her face with water and dries her face off with a towel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}