{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"de37c2f0-35fb-5f14-8723-5efd7d1af120","task_key":"default--validation--16995","task_revision_id":"1","upstream_id":"16995","short_description":"HellaSwag validation 16995","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A man is walking in a field of honey bee boxes. He begins to use a weed eater to trim the weeds of the field. he\",\"endings\":\"['spins the plant eater on his bynard as he walks back and forth.', 'shear the weeds evenly, as he lays back and then begins smoking.', 'uses an ax to cut the trimmed weeds.', 'finishes and walks off the field.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}