{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9c1ad10-b305-544b-9140-54e038309280","task_key":"default--validation--6159","task_revision_id":"1","upstream_id":"6159","short_description":"HellaSwag validation 6159","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A young man stands, at night, in shorts and does brief stretches before getting a running start and running towards a raised bar. the man\",\"endings\":\"['does a trick high on his bar, performs a fast warm up run, flames appear in his footwork and the first man offers him a number of times on trampoline till he reaches the end and drinks a lime.', 'lets go of the bar and vaults into the air, holding securely on to its bar, releases the bar and serves it in an elaborate round of tangled, shimmy.', 'hurls himself in the air and jumps over the raised pole backwards after which he lands on a thick red gym mat on the other side.', 'performs high jump and circles his legs before getting on the bar.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}