{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9f97bef-4eae-5bc1-9346-061d01863f85","task_key":"default--validation--26560","task_revision_id":"1","upstream_id":"26560","short_description":"HellaSwag validation 26560","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to avoid peeing on the tampon string [title] Sit on the toilet seat, but don't pee yet. [step] If you are using a public toilet, you may wish to squat above the toilet, without actually resting on the seat. Otherwise you can use a seat-liner or tear several strips of toilet paper, lay them on the toilet seat, and sit.\",\"endings\":\"[\\\"[title] Use a toilet brush to gently brush away any contaminant from the brush. [step] This is important so that the brush doesn't have an unsightly dislodge from the string.\\\", '[substeps] Make sure you have pulled down your pants and underwear or lifted your dress or skirt before sitting down. Squeeze the muscles around your urethra (the opening through which urine exits your body).', '[substeps] If you have a notepad, a pen, or a sharpie pen, use those to jot down notes. [title] Take some time to conduct research on tampons.', '[substeps] If you don\\\\'t feel comfortable doing this yourself, see someone who can do it. You might tell them, \\\" it\\\\'s okay to look for toilet paper.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}