{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f34f10e-5488-5bed-8564-d68a36aaf45a","task_key":"default--validation--26474","task_revision_id":"1","upstream_id":"26474","short_description":"HellaSwag validation 26474","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to remove super glue from your skin (petroleum jelly method ) [title] Assess the situation. [step] So long as the super glue isn't on or inside of your eyes or inside of your mouth, you can safely remove super glue at home. If you have super glue on any of these other regions, see your doctor or visit an emergency room or urgent care as soon as possible.\",\"endings\":\"['[title] Dab some petroleum jelly onto your wrist. [step] Carefully grasp your wrist with one hand while applying super glue with the other.', '[substeps] A doctor can most effectively remove super glue from the sensitive eye areas. In most cases, super glue will solidify on contact with saliva.', '[title] Place your hands on someone who has super glue on their skin. [step] Use a few layers of petroleum jelly to clean, exfoliate, and press the super glue.', '[title] Dab a hydrogen peroxide solution on as many problem areas as possible. [step] Hydrogen peroxide helps to break down super glue on skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}