{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac6b85ce-dcc6-5900-8410-4552a881986d","task_key":"default--validation--7870","task_revision_id":"1","upstream_id":"7870","short_description":"HellaSwag validation 7870","config":"default","split":"validation","body":"{\"activity_label\":\"Raking leaves\",\"ctx\":\"A man rake dead leaves in a backyard. then\",\"endings\":\"[', the man slides on the sidewalk, then approaches the fallen leaves with a rake and collect them.', ', the man turn the mower around and continues rake the leaves.', ', the man moves jumping the buckets to reach large bushes.', ', the man put the leaves in a bags.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}