{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29e1393f-ca65-595c-a468-e3f9bbf1306a","task_key":"default--validation--41261","task_revision_id":"1","upstream_id":"41261","short_description":"HellaSwag validation 41261","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"He lifts the weight and first brings it parallel to his shoulders. He then lifts it completely over his head with his arms extended. after\",\"endings\":\"['leaving his arms up for several seconds, he allows the weight to fall on the ground.', ', he lifts the weight over his head, and a hole starts to appear in the middle of his chest.', 'he drops the weight from his hands, he stands up and shows us the weight.', \\\"he drops the weight, another man appears and he's shirtless and hiking.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}