{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a134b7c3-2501-5804-b901-f2d2d7d97845","task_key":"default--validation--38411","task_revision_id":"1","upstream_id":"38411","short_description":"HellaSwag validation 38411","config":"default","split":"validation","body":"{\"activity_label\":\"Pole vault\",\"ctx\":\"The man catapults into the air, releases the pole and goes over an elevated line measure bar, touching the bar with his body before landing on the landing cushion underneath the bar. the man\",\"endings\":\"['falls to the cushion with a look of disappointment, stands up, with the appearance of dejection, but smiling at the same time, while walking away from the cushion and picking the pole vault back up.', 'throws his body into a howcast pose.', 'dismounts from the diving rope but his legs have hung off the board and is swaying out in the air a bit.', 'recovers from his landing and walks to the edge of a platform before bungee jumping.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}