{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6e682da-4446-5302-aa4a-1ced6daf9800","task_key":"default--validation--6415","task_revision_id":"1","upstream_id":"6415","short_description":"HellaSwag validation 6415","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to change a disposable diaper [title] Lay the child on a flat surface. [step] Make sure the surface is clean, dry and not cold to the touch. Stand at the edge closest to your child's feet, with them lying down in front of you, their toes closest to your body.\",\"endings\":\"['[substeps] You should also sit on the edge to their right or left side. If possible, one person should sit on your right side, and the other should sit on your left side.', 'Make sure they are not kicking their legs up in the air. [title] Pick up the object, which should be the tube that continues through the hand of the baby.', 'Remove any clothes that will impede the changing of their diaper. [substeps] After laying your child down, wait for a second or two before beginning to change their diaper.', \\\"[substeps] Don't touch their head, chin, or mouth unless you're younger and you've just gotten out of a dirty diaper. Wash your hands if you're changing from the inside out.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}