{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38156830-3963-5f05-810f-ff99f68ecfe2","task_key":"default--validation--23898","task_revision_id":"1","upstream_id":"23898","short_description":"HellaSwag validation 23898","config":"default","split":"validation","body":"{\"activity_label\":\"Slacklining\",\"ctx\":\"A tight rope is strung across the length of the pool and a woman is attempting to walk across, she falls several times. the last time she falls\",\"endings\":\"['she gets up and stands on edge of the pool, she tries to pole vault into the pool.', ', the camera rewinds and shows her falling backwards.', 'down, she tries to look across but the rope is too tight.', ', the man shoots her several times before he stops her.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}