{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05a3211b-5f3f-5775-953b-268b84f80b69","task_key":"default--validation--50390","task_revision_id":"1","upstream_id":"50390","short_description":"HellaSwag validation 50390","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for kid's dancewear [title] Keep dance shoes dry. [step] Shoes have a tendency to fray, especially satin shoes. To prevent shoes from fraying, avoid getting your shoes wet.\",\"endings\":\"['Instead, remove them every two to three hours or every day until they are dry to the touch. [substeps] Even if your shoes do not need to be wet, keep them relatively dry at all times.', 'Apply a dust mop on a 2-inch (5.08-centimeter) wide section of your shoe. Spray water over the entire shoe before wiping it down to make sure that it is dry.', \\\"Hand wash socks or other non-slip types of shoes when they're dry. [substeps] You can find dry socks at specialty shoe stores.\\\", \\\"Make sure your kids do not walk around in their shoes while it's raining. Avoid leaving your shoes anywhere they may accidentally get wet, such as near the front door.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}