{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f1d00804-01d2-51ce-8830-6f06e65d886f","task_key":"default--validation--19072","task_revision_id":"1","upstream_id":"19072","short_description":"HellaSwag validation 19072","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A woman raise a weight until her shoulders, then she steps out a foot and raise the weight above her hear. then\",\"endings\":\"[', woman raises the weight seen from above until the shoulders.', ', woman lift a weight, placed it on the ground and walk, then the woman drop the weight on the ground and bow by bending her hip.', ', the woman drops the weight to the floor.', ', the woman raises the weight to her chest.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}