{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af3d762e-acb6-5395-a38c-aa8bd5ac0739","task_key":"default--validation--39315","task_revision_id":"1","upstream_id":"39315","short_description":"HellaSwag validation 39315","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"A man jumps rope in a boxing ring in a mall as people watch. The man loses his rhythm and has to start over two times. the man\",\"endings\":\"['stops jumping and his coach pulls the tape off his hands.', 'goes over three times in circles but eventually wins, the man is beside him on the dirt.', 'makes his way to the finish line checking his score as people walk by.', 'attempts to fix the sin as he goes back and forth between rounds.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}