{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b654030d-6c63-55bf-8d02-00ca84274b90","task_key":"default--validation--17509","task_revision_id":"1","upstream_id":"17509","short_description":"HellaSwag validation 17509","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to repair silver jewelry [title] Get two pairs of flat nose pliers. [step] You can buy these for relatively cheap at your local hardware store. You can also purchase a jewelry repair kit from a hobby store, which will come with special pliers and tools needed for repair.\",\"endings\":\"[\\\"[substeps] If you're going to be doing multiple pieces at once, you'll need a pair of back-cut pliers, so you only need two pairs instead of three. [title] Find a sharpie to shade the silver.\\\", '[substeps] Usually, the kit includes 17 or 24 small flat head pliers designed to help mend jewelry. The kit will also have arm wrenches and socket heads available.', \\\"Just make sure you get a pair of flat nose pliers that are small enough to handle smaller pieces of jewelry. [substeps] If you're buying the pliers on their own rather than as part of a kit, you'll need two pairs in order to hold each end of the broken clasp.\\\", '[substeps] You want to replace silver jewelry pairs every two months in order to maintain its luster. If you want to take it to a professional, their equipment can make it easier for you to do so.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}