{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9d691de-9129-5543-aa73-eb82a2187e02","task_key":"default--validation--19327","task_revision_id":"1","upstream_id":"19327","short_description":"HellaSwag validation 19327","config":"default","split":"validation","body":"{\"activity_label\":\"Archery\",\"ctx\":\"A man is seen speaking to the camera while showing various tools to the camera as well as a close up of a bow and arrow. The man shoots the bow while showing it to the camera several times and shooting more. he\",\"endings\":\"['continues speaking and shows off his bow in slow motion and wiping his hands off.', 'finally finishes by speaking to the camera.', 'continues speaking while showing off his arrows and leads into him using the furniture as a backdrop.', 'points to another object before finally presenting it to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}