{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab0bf4c9-3ce8-5202-bb53-c67771581fdf","task_key":"default--validation--33167","task_revision_id":"1","upstream_id":"33167","short_description":"HellaSwag validation 33167","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"A man in a grey track suit performs jump rope tricks in front of a group of students. the man in the grey track suit\",\"endings\":\"['finishes and the children clap.', 'performs jump rope tricks indoors on a wooden stage with the audience watching in support.', 'successfully jumps free from jump rope to comes back to the rope and repeats the jump rope tricks in reverse while demonstrate multiple jumps with different people walking onto the rope.', 'spins on the rope with close ups of his feet as he performs flips the rope.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}