{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3fa4960-978e-556d-b663-0eb0001a31a1","task_key":"default--validation--3889","task_revision_id":"1","upstream_id":"3889","short_description":"HellaSwag validation 3889","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to do a back handspring [title] Bend your hips forward and swing your arms down. [step] To begin, stand up straight with your arms extended by your ears. Then, sit back, like you're going to lean back in a chair.\",\"endings\":\"['As you do this, lift your arms straight up into the air, like you\\\\'re going to say \\\" hoe-bee-bee! \\\". [substeps] It\\\\'s pretty simple to do, but it will come with a bit of repetition.', 'Remember that your hands should be just slightly bent with your palms resting on the floor. [substeps] To do the front handspring, stand with your feet hip-width apart.', 'Your shoulders will form a right angle to the ground. [substeps] If you are in a chair pose, rotate your pelvis to one side as you roll your body forward.', \\\"As you do this, swing your arms down and back behind you. Your knees should be directly over your feet, not positioned in front of your feet, or you won't generate enough momentum to land on your hands.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}