{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed36d725-0d71-5c8f-9b12-3aad0b20aba2","task_key":"default--validation--19026","task_revision_id":"1","upstream_id":"19026","short_description":"HellaSwag validation 19026","config":"default","split":"validation","body":"{\"activity_label\":\"Disc dog\",\"ctx\":\"A man is standing on a field, playing with a dog and a frisbee. Several people are seated all around, watching. the dog\",\"endings\":\"['runs across the grass, forcing the man to run after it.', 'lunges for the frisbee and the man catches it.', 'finishes the frisbee and the man throws it at the next person walking by.', 'chases and catches the frisbee numerous times.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}