{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0143807-6cbe-5e52-9a55-fac05f36f0c9","task_key":"default--validation--35429","task_revision_id":"1","upstream_id":"35429","short_description":"HellaSwag validation 35429","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to develop a ninja like physique [title] Clear a space on the floor, large enough for you to lay down in. [title] Start stretching your legs by sitting on the ground, with your legs in a v shape. [title] Push them as far as part as you can without hurting yourself too much.\",\"endings\":\"[\\\"[step] You should fell a little bit of a stretch, but it shouldn't hurt too much. [title] Keeping your back as straight as possible throughout, reach for your right foot first and hold it for 30 seconds, then reach for the other foot.\\\", '[title] Use a barbell to elevate yourself up. [title] Use the muscles in your legs to push against the wall.', '[title] Slowly begin to push your legs flat. [title] When you feel your toes begin to put weight on them, stand up and gently push your feet off the ground and lean back on your arms to take your weight off them.', '[step] Once you have done this, give yourself a little more stretching time to get used to the feel of training. [title] Stretch your arms by gently pulling them out of the v-shape to keep them in place.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}