{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c9aedf07-8d55-578e-b89f-dc24f13ca5b8","task_key":"default--validation--13548","task_revision_id":"1","upstream_id":"13548","short_description":"HellaSwag validation 13548","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a blanket scarf [title] Tie your scarf into the looped style. [step] The \\\" loop \\\" is one of the most popular and traditional ways to tie a blanket scarf. Fold your scarf in half.\",\"endings\":\"['Gather it in the center and place it around the back of your neck. Allow one side to hang down a little more than the other by about four to five inches.', \\\"Tie the knot at the base of the scarf so it doesn't bunch up in the middle and add a bit of extra. Lay the scarf flat on your work surface.\\\", 'Slide the scarf over one of your ends to form an l-shape. Pull your hair over your hand to wrap your scarf around your head.', \\\"Place the left end of the scarf diagonally on your shoulder, wrap the right end around your neck and tie the two ends together. Once you've looped, tie the right end of the scarf into the loop on your left arm.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}