{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8fc111aa-4eb4-5585-b761-7eb0b87e7131","task_key":"default--validation--32318","task_revision_id":"1","upstream_id":"32318","short_description":"HellaSwag validation 32318","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man walks up to a barbell and bends over, lifting at the knees. he\",\"endings\":\"['brings his hands behind his back as he lifts himself up, lifting the weight over his head.', 'lifts it over his head, throwing it to the side, and then stands again for a moment.', 'walks out of view, and a man wearing a white shirt is lifting weights at the barbell station.', 'lifts the barbell up to his chest, rests for a moment, then lifts it over his head.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}