{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"999e4e3b-549a-5cfc-a2a0-069f26a29e3c","task_key":"default--validation--22511","task_revision_id":"1","upstream_id":"22511","short_description":"HellaSwag validation 22511","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"One of the body builders lifts the weight bar several times and then drops it down while the other two body builders watch him. then he\",\"endings\":\"['goes and wipes off his sweat with a towel and lifts the weight bar again and drops it.', 'releases the weight bar, as well as the beam he was holding, and drops it down near the floor.', 'continues to lift for another couple of seconds and then drops it and the body builders fall.', 'runs towards the weight as the others fall with the weight and watch him drop down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}