{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"288af79a-1194-553d-9f75-e22f6bb275d1","task_key":"default--validation--21898","task_revision_id":"1","upstream_id":"21898","short_description":"HellaSwag validation 21898","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A woman is seen standing before a camera and begins putting lotion on her hands and rubbing it round her face. the woman\",\"endings\":\"['wipes the lotion all over her face while still speaking to the camera and leads into her speaking to a girl in the distance.', 'continuously rubs lotion all into her face and ends by looking into the camera.', \\\"holds up a hair dryer and begins curling around the woman's hair while she continues to lay down and standing in the end.\\\", 'then brushes past the camera and begins smiling to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}