{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d5b38c6-9983-58e2-be66-3caa27f7f82f","task_key":"default--validation--38410","task_revision_id":"1","upstream_id":"38410","short_description":"HellaSwag validation 38410","config":"default","split":"validation","body":"{\"activity_label\":\"Pole vault\",\"ctx\":\"An athlete pole vaults at a professional sports event in front of an audience in bleachers and sports event workers and spectators, all in slow motion. An athlete runs straight head while holding a long pole vault with both hands during the run. the man\",\"endings\":\"['vaults over a bar on a tennis court.', 'catapults into the air, releases the pole and goes over an elevated line measure bar, touching the bar with his body before landing on the landing cushion underneath the bar.', 'vaults at parallel bars in slow motion while holding a rope.', 'carries the long pole vault with him where he does this twice before showing the finished throw in slow motion.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}