{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cadddf2-f007-5d9b-a5ad-132de658c25f","task_key":"default--validation--7997","task_revision_id":"1","upstream_id":"7997","short_description":"HellaSwag validation 7997","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"After, the woman takes out the container of the mower and puts the dead grass in a trash bag, then she puts back the container in the mower. a boy\",\"endings\":\"['is guiding the girl with a pick.', 'makes a sweat log, then he does the same, after the woman begins to mow around the yard area, the boy come on to help.', 'stands watching the woman.', 'runs using the mower behind the house and high kicks some grass, then he goes to play with other boys.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}