{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbbba4fc-bd9c-5806-a912-5ec098dac2c9","task_key":"default--validation--28337","task_revision_id":"1","upstream_id":"28337","short_description":"HellaSwag validation 28337","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean an eyebrow piercing [title] Wash your hands before cleaning your piercing. [step] Never attempt to clean a piercing without washing your hands first. Bacteria from your hands can easily rub off onto the piercing, so wash your hands thoroughly with soap and water before beginning the cleaning process.\",\"endings\":\"['[substeps] Wash your hands with clean, running water and soap for 20 seconds. To keep track of time, hum the \\\" happy birthday \\\" song to yourself twice.', '[title] Clean the piercing with hand sanitizer. [step] Before you can clean the piercing, you must make sure that you have properly sanitized your piercing.', 'You may also want to wash your hands under running water. Before cleaning, wash your hands to remove any germs or dirt.', 'Use enough soap to dampen your fingers and scrub your piercing with the water. Begin at the piercing and dab a bit of water onto each finger, feeling for any dirt or debris.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}