{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef06a48c-4501-5dba-875d-b25f772920d0","task_key":"default--validation--22510","task_revision_id":"1","upstream_id":"22510","short_description":"HellaSwag validation 22510","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"There are three body builders exercising and working out in a small gym with weightlifting equipment. one of the body builders\",\"endings\":\"['is walking around in a black t made workout outfit.', 'who is wearing black, lifts the other guy while they continue working out.', 'lifts the weight bar several times and then drops it down while the other two body builders watch him.', 'begins to lift a barbell and the other two male take care to side step back and forth as their posture makes it harder for them to lift it over their heads.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}