{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2fed5f93-4037-5d46-b73e-d3c2dffc28a3","task_key":"default--validation--7145","task_revision_id":"1","upstream_id":"7145","short_description":"HellaSwag validation 7145","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to do leg lifts [title] Lie flat on your back with your legs stretched out in front of you. [step] Your legs should just be a toe's width apart. Make sure to keep your hands down flat on the ground near your sides, with your palms down.\",\"endings\":\"['[substeps] You can use a yoga or exercise mat for extra support and comfort. If you experience occasional back pain, you can roll a towel and place it under the arch of your back, just above your hips.', '[substeps] Your arms should span on either side of your body, between the top of your head and the back of your neck. Your torso should be resting on the floor between your hands and your shoulders.', 'Keep your feet on the floor just a little wider than shoulder-width apart. [substeps] You may also consider digging your fingers into the floor and bringing your feet closer together, allowing them to contract as you tense your muscles.', \\\"[substeps] Keep your heels on the floor about an arm's length apart, or approximately 22 inches, or as much as two. Your feet should make up the majority of your body weight.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}