{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af500b48-cdc3-5848-9482-b271c1f3c273","task_key":"default--validation--39843","task_revision_id":"1","upstream_id":"39843","short_description":"HellaSwag validation 39843","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to figure skate (for beginners ) [title] Gear up. [step] Before you get out on the ice, you need to put on your skates. Don't worry about buying your own gear right away.\",\"endings\":\"['You need to buy the right skates. Or else you run the risk of ruining your skates.', \\\"You'll be carrying it out at your skating rink, so it's a good idea to save it for later. Whether on you skates or out on the ice, it's important to make sure you're warm and comfortable.\\\", 'There are some tips to keep in mind when assembling your skates : [substeps] Beginners need to make sure that your skates are in good condition before attempting to take them out. If your skates are worn out, you may not be able to skate well in them.', \\\"You can rent your gear until you're sure you want to figure skate for a long time. Remember to dress warmly! Gloves and scarves are good.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}