{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c022fca-042b-5f26-a5c1-82ccf2770f81","task_key":"default--validation--7424","task_revision_id":"1","upstream_id":"7424","short_description":"HellaSwag validation 7424","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man lifts a barbell with weights from the ground up in a gym with other people nearby. the man\",\"endings\":\"[\\\"slowly lifts his weight doing squats on front a barbell and looking back down after he's done lifting it.\\\", 'lifts the barbell continuously with one hand upwards placed on his shoulders.', 'lifts the barbell up above his head.', 'begins to lift the barbell through the air and puts it on his head.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}