{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6f56441-345a-5c60-8ae7-55e65992fa31","task_key":"default--validation--36677","task_revision_id":"1","upstream_id":"36677","short_description":"HellaSwag validation 36677","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"After that, she doesn't say anything else and the boy is seen pushing the lawnmower in different directions and the woman speaks again. in the end\",\"endings\":\"[', the boy is tired and is still mowing the lawn while the woman says good job desmin.', ', the person is talking to the camera and begins to bring out the lawn mowing equipment to the dog from the driveway.', ', he moves all the items and the woman continues driving while a young man shows it to the camera.', ', she puts down the sickle and walks over to the boy and adds another to the field for added weight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}