{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b10fad71-d46c-5c30-bf5e-af0ec3559566","task_key":"default--validation--36489","task_revision_id":"1","upstream_id":"36489","short_description":"HellaSwag validation 36489","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to do an elbow stand [title] Find an open space on flat ground. [step] To get into an elbow stand comfortably, you're going to need some space. Find an open, flat section of floor that's big enough to accommodate the length of your body.\",\"endings\":\"['[title] Take your right foot off of the ground. [step] Your left will be a bit larger for your body to stand on.', \\\"If you can't find a ground that's big enough, you can find a platform/wall nearby. [substeps] If you can't find a comfortable area, put something sturdy on the ground before you start the exercise.\\\", \\\"That way if you lose your balance while holding the elbow stand, you won't have to worry about colliding with any obstacles nearby. [substeps] Start learning the elbow stand on a carpeted floor, or outside on grass.\\\", \\\"That's where you'll place your hands and feet. [substeps] To make sure your body has room to move freely, it's best to practice arm and leg labor with a partner or with a coach/friend.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}