{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d47ade3-6a73-5823-8970-6be3fb1699b6","task_key":"default--validation--9111--variant-eebbf95d36dd","task_revision_id":"1","upstream_id":"9111--variant-eebbf95d36dd","short_description":"HellaSwag validation 9111","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to make your locker useful and decorative [title] Clear your locker if it is messy. [step] If it's brand new, you are very lucky! New lockers are the best. No mess, no fuss, no worry over what there is to throw out and what not.\",\"endings\":\"['Clean with soap and water and rinse well. The key is to clean the outside of the locker.', '[title] Purchase things that you think you should have. [step] Items such as locker shelves and magazine racks are perfect for spaces as small as a locker.', \\\"[title] Pick the best editor that would edit the locker in an insightful way. [step] To get a good editor, go to schools' website, and you can show them your locker is the one with the best information.\\\", \\\"All that really costs we are really looking to get rid of is clutter. [title] Clear it! [step] Maybe you wish you had something from your locker where you know you'll remember what.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}