{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30868a35-3cce-58a4-ac1f-99179b0feae3","task_key":"default--validation--48178","task_revision_id":"1","upstream_id":"48178","short_description":"HellaSwag validation 48178","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A very thick man walks slowly into a a large room filled with people and he stands in front of a barbell with red and blue weights on both ends. The man takes a few breaths then reaches down and puts his hands on the barbell. after a\",\"endings\":\"['long time, he grabs the edge of the barbell and lifts it up putting his small hands on it.', 'second, the man stands straight up and the group of people start to cheer and high five, one of the men then jumps and high fives the other two people.', 'few seconds, he looks up and smiles in triumph.', 'while the man lifts the barbell to he neck, then jumps and bends his legs at the same time putting the barbell above his head, then he throws the barbell to the ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}