{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7736ec6e-0ce0-5858-91e0-c480322e51df","task_key":"default--validation--36126","task_revision_id":"1","upstream_id":"36126","short_description":"HellaSwag validation 36126","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"The baby is looking at the camera and then the hand wiped his face. The hand wiped the baby's face and the baby opened his mouth. the hand\",\"endings\":\"['threw the frisbee and the baby grab it and as he throws it, it flew through the air and he finally caught the frisbee.', 'started to wipe the step with the cloth.', \\\"wiped the side of the baby's face and nose and the baby clapped.\\\", \\\"bring out toothpaste and wash the baby's face to clean it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}