{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae2fff32-2fc7-58a3-a4d4-6848c306c095","task_key":"default--validation--28136","task_revision_id":"1","upstream_id":"28136","short_description":"HellaSwag validation 28136","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A man is shown mowing his lawn on a hazy day. He goes back and forth and looks into the camera and seems to be struggling. he\",\"endings\":\"['goes back and forth making sure to stay in a straight line.', \\\"continues his mowing and looks into the camera leaves but joins another man when he's done.\\\", 'turns and smiles to the camera as he continues to mow his lawn.', 'closes the mower and wanders back and forth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}