{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11abb0ac-a7cf-5971-a2ee-868e42e2e29c","task_key":"default--validation--34097","task_revision_id":"1","upstream_id":"34097","short_description":"HellaSwag validation 34097","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to fix painful shoes [title] Prevent blisters, chafing, and cutting by sticking pieces of moleskin inside your shoe. [step] Purchase some moleskin from a shoe store (or the shoe repair section of a drug store) and take out one sheet. Place the sheet behind the offending strap or heel and trace it with a pencil.\",\"endings\":\"['Repeat the same process on the rest of the bad or painful area. [substeps] Out with your shoes and have someone else check the bottom of the back of them.', 'Cut the shape out using a pair of scissors and peel off the backing. Stick the moleskin onto the strap or heel.', 'Place the sheet wherever possible and spread the moleskin out evenly across the entire shoe. Adjust the sheet every so often, so that it covers as much of the raw part of the shoe as possible.', \\\"Don't forget to press down firmly on the moleskin with your pencil. Wrap a rubber band around the padding/pleat pad that attaches the pad to the toe of your shoe/heel.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}