{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67adb457-13fe-5e19-a055-da4865cba506","task_key":"default--validation--25238","task_revision_id":"1","upstream_id":"25238","short_description":"HellaSwag validation 25238","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man ina shirt that says beast mode is making a video on what the best free workouts to do without a gym are. He has a weight bar with no weight and he bends over a certain way. he\",\"endings\":\"['shows you how to use the weight bar for different weight lifting positions.', 'also has words on the screen on how to workout as well as how to talk about the correct way to back out of it.', 'begins to skip across the weight bar, trying to ignore the pain, until he briefly stops.', 'then takes off the weight as well as throws it down and starts talking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}