{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1e9087b-ef1c-5b81-936d-4232be6fc76a","task_key":"default--validation--2308","task_revision_id":"1","upstream_id":"2308","short_description":"HellaSwag validation 2308","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to walk down stairs in high heels [title] Know how to walk in heels confidently. [step] Try to practice as often as you can, even in your room, until you can walk without stumbling. This is very important when you have to cross a surface that isn't flat.\",\"endings\":\"['[title] Have extra caution and have a straight posture. [step] When you first start walking in heels you may wish to be extra careful and watch your feet, but ultimately you need to learn to keep your head high, shoulders back, and back straight to reap the full benefits.', '[title] Place your hand on your heel, your heel higher than usual, and push into your heel toward your hand. [step] Try not to bang your heel down too hard, because then you will hit yourself in the leg area where you are standing.', \\\"If you have a tendency to stumble at any time, however, practice your walking in heels whether you're walking on a solid floor or on an elevator. [substeps] Never put one foot on the ground while walking down a flight of stairs, and never let your heel touch any ground.\\\", \\\"If you find yourself walking down stairs where there are stairs that you won't be able to walk up, stretch your legs out in front of you. [substeps] When balancing on your heels, keep your wrist at a ninety degree angle to the ground, and keep your foot in that position for 1 second.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}