{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d7443077-8854-5343-a590-0d967fcd15b3","task_key":"default--validation--13407","task_revision_id":"1","upstream_id":"13407","short_description":"HellaSwag validation 13407","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to paint shoes [title] Use leather or spray paint for leather or vinyl shoes. [step] There are acrylic paints designed to adhere to leather products, including shoes. You can buy them at your local craft store.\",\"endings\":\"['[substeps] Cover your shoe with a piece of rubber cling film after pouring in the paint. Allow it to dry completely before walking barefoot on it.', 'These paints go on with a brush for a smooth and lasting finish. Your other option is to use spray paint purchased from a hardware store.', 'Some even come with extra spraypaint depending on the design, so that you can paint the insole, parts, or strap of the shoe. [title] Apply anhydrous adhesive to the insole.', \\\"If you decide to use paint, you'll need to consider washing your shoes before use. This should ensure that the paint doesn't damage the leather.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}