{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0821742b-7381-5712-8272-0476048dc8d2","task_key":"default--validation--21289","task_revision_id":"1","upstream_id":"21289","short_description":"HellaSwag validation 21289","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean white converse [title] Remove the shoelaces. [step] Unlace your shoes to expose as much of the white canvas as possible, making it easier to thoroughly clean your sneakers. [substeps] You can clean shoelaces separately by letting them soak in a bucket or basin of warm, soapy water, but know that they may never be as bright as they once were.\",\"endings\":\"['Soaking in the water, rub it between your hands, and clean the converse thoroughly on a towel. [title] Decide whether you want bleach or water stain remover.', \\\"[title] Disinfect a brush with bleach if desired. [step] Take a small microfiber cloth, dip it in a bucket of bleach, and wipe down your white canvas surface while it's still wet.\\\", \\\"[title] Wipe the sides of sneakers with a clean, damp cloth. [step] After you've soaked your entire shoe in the bucket of water for several minutes, take the damp cloth and wipe it over the surface of your converse.\\\", 'An alternative is to replace your old shoelaces with new ones. [title] Rinse the shoes under running water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}