{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35bc6ebe-bb53-5b32-9c86-d2a2d32e26f4","task_key":"default--validation--12251","task_revision_id":"1","upstream_id":"12251","short_description":"HellaSwag validation 12251","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to do a front flip on the trampoline [title] Start by doing handstands on your trampoline. [step] When doing the handstand fall over onto your back or bottom. Keep doing this for about 2 days.\",\"endings\":\"['[title] Do one handstand at a time. [step] All you have to do is do a handstand try to do at least 4 handstands on the trampoline at a time, with about 2 side flips each.', '[title] Continue doing handstands and falling over, but do them faster. [step] That means go on your hands then fall over as quickly (and safely) as possible.', 'Work up your initial grip, then for more you will be going over your hand and arm holds. [title] Using both of your arms as leverage, tap your hands on the trampoline, moving them from side to side and move them all up in an \\\" l \\\" shape.', 'You can go on more than one handstand. Now : [substeps] Get your heel on the trampoline.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}