{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db22d449-d50a-5825-b4ba-1ab53d3339b0","task_key":"default--validation--23408","task_revision_id":"1","upstream_id":"23408","short_description":"HellaSwag validation 23408","config":"default","split":"validation","body":"{\"activity_label\":\"Hand washing clothes\",\"ctx\":\"A person puts a garment in a pail, then he adds detergent and water. then\",\"endings\":\"[', the man washes the cloth with his hand, and then pours water and rinse several times.', ', the person holds the pail and pours the detergent in the cloth.', ', the man dries the pail in a dryer.', ', the person puts fresh cleaning solution and then cleans with his hands.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}