{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3def7c8b-5e47-5d32-9973-d435c4d0fb4b","task_key":"default--validation--49682","task_revision_id":"1","upstream_id":"49682","short_description":"HellaSwag validation 49682","config":"default","split":"validation","body":"{\"activity_label\":\"Spinning\",\"ctx\":\"A woman went down from the stairs walked towards the cycling machine then started to pedal. the woman\",\"endings\":\"['changed her mind and continue pedaling in a straight line and did a slow handstand then she finish while having a laugh.', 'go down from the floor then go up and start the machine.', 'went down the stairs and again went down the stairs and started to pedal.', 'adjusted the chair by twisting the screw and then she pedals again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}