{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f753dd4-599f-5989-a3ed-58633ee82a32","task_key":"default--validation--4998","task_revision_id":"1","upstream_id":"4998","short_description":"HellaSwag validation 4998","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A gymnast bend to rise weight above his head, while other men sit in the gym watch him. the gymnast\",\"endings\":\"['holds the weight bordering his sit position and runs on the distance.', 'drops the weight to the floor.', 'performs a flat weight fall in the gym instead, then also he stand and use hands to support himself, he does the horizontal balance, then he flips and lands in the gym next to other men.', 'performs a high jump successfully.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}