{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3fdd4d69-3d40-5674-b7c0-de8ec327d0c6","task_key":"default--validation--22178","task_revision_id":"1","upstream_id":"22178","short_description":"HellaSwag validation 22178","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to do spiderman planks [title] Begin on your hands and knees. [step] Get into a solid plank position by spreading your hands shoulder width apart on an exercise mat. Place your knees on the mat, directly under your hips.\",\"endings\":\"['[substeps] Maintain a v-formation, bending your right knee and reaching your left elbow over your left knee. Rest your palms on the mat for balance.', 'Make sure your weight is evenly distributed between your knees, your knees easily sitting on either side, and your arms straight up. [substeps] Any time someone tells you to do a spiderman plank, get into an easier plank position.', 'Raise your arms above your head in a traditional \\\" duck \\\" position. [substeps] You could also assume a bunny-cow pose.', 'Roll your shoulders down your back and put equal weight in both palms. [substeps] Starting on your hands and knees can allow you to set up the plank position correctly.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}