{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1326078-1830-5cf0-a9c1-d3cbc068f243","task_key":"default--validation--24582","task_revision_id":"1","upstream_id":"24582","short_description":"HellaSwag validation 24582","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean keds [title] Remove laces and inserts. [step] Before you clean your keds, you should take out both the laces and any inserts inside of the shoe. The laces can be cleaned in the washing machine, by hand in the sink, or simply replaced.\",\"endings\":\"['[title] Soak your keds in water for 10 minutes. [step] You can use your cleaning solution-soap, warm water, or any liquid that you like.', 'You should set aside the inserts until you are finished cleaning. If they are smelly or dirty, you may want to replace them.', '[title] Soak the shoelaces and inserts. [step] Place the lace and inserts inside a bowl or cup and gently soak them in a sponge full of water.', \\\"[substeps] This step will be optional, but may be recommended for those keds where you're cleaning larger packages or clothing items. Dry your keds with a microfiber cloth.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}