{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5247388-35ee-5ed3-87d8-eeb7e2ac6dff","task_key":"default--validation--19326","task_revision_id":"1","upstream_id":"19326","short_description":"HellaSwag validation 19326","config":"default","split":"validation","body":"{\"activity_label\":\"Archery\",\"ctx\":\"A man is seen speaking to the camera while showing various tools to the camera as well as a close up of a bow and arrow. the man\",\"endings\":\"['then begins using the bow on several targets and continues explaining around them while moving all around.', 'shoots the bow while showing it to the camera several times and shooting more.', 'then sharpens the bow and begins to shoot it while others walk in and out of frame in the back.', 'then draws a face with a whetstone over and over on the wall while moving the sharpening stone around.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}