{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"767447ff-8c5f-5287-a44f-468fb5d3781e","task_key":"default--validation--18149","task_revision_id":"1","upstream_id":"18149","short_description":"HellaSwag validation 18149","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"The girl is on front a sink and puts soap on her face, then she remembers the two girls splashing water on her face. then\",\"endings\":\"['she dons a wrong shoe and falls during her fall on the floor.', 'she goes to the kitchen area in some small fenced in area.', ', the girl is washing the hands under the hot boiling water, then she wash the stomach and the hands together.', ', the girl takes water with both hands and splash on her face and some water spills on the floor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}