{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04d740c4-f016-567f-9c07-f464aadf5896","task_key":"default--validation--3193","task_revision_id":"1","upstream_id":"3193","short_description":"HellaSwag validation 3193","config":"default","split":"validation","body":"{\"activity_label\":\"Camel ride\",\"ctx\":\"A woman rides a camel holding a rode with her right hand while a man pulls the camel. then\",\"endings\":\"[', the camel stop, and a person carry guide in his hands, then they ride on its back and return.', ', the woman stop walking and and retrieve the camel.', ', the camel picks up the woman and hold her while she looks behind while walking.', ', the camel stops and the woman gets down from the camel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}