{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae3c49fd-6c09-5ffa-a76b-4d6dfc18a83c","task_key":"default--validation--22626","task_revision_id":"1","upstream_id":"22626","short_description":"HellaSwag validation 22626","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"The sun peeks from behind the clouds and lights the scene up. The man stops and empties grass in the garbage. A man enters carrying a baby and leaves. the man\",\"endings\":\"['throws a frisbee for the baby to catch.', 'mows along side of the bushes.', 'demonstrates cutting the grass around the crib.', 'grabs the baby who then gets up on his shoulders and twirls him around on the grass.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}