{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1e200563-9c6b-5264-81cb-07e0df5cbca5","task_key":"default--validation--29199--variant-7645e2a7b226","task_revision_id":"1","upstream_id":"29199--variant-7645e2a7b226","short_description":"HellaSwag validation 29199","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean under acrylic nails [title] Wash your hands with antibacterial soap and warm water. [step] Do this whenever you go to the bathroom, eat or cook, touch animals, or handle dirty objects. Don't wash your hands unless they need it, however.\",\"endings\":\"['[substeps] Make sure to be gentle on your hands. Wear gloves when washing or cooking non-stick surfaces because nonstick varnish oils can easily fall off.', 'Too much washing can weaken the nail glue. [title] Dry your nails thoroughly with a towel whenever they get wet.', 'That way, if you end up with a rash, there is a fine line between bacteria and dirt. [substeps] Avoid using water that is too hot or too cold on your hands, which can damage them.', 'Dirty hands automatically transfer bacteria to other nails. [title] Scratch the inside surfaces of the nails.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}