{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8462d7a0-1b17-5bcc-9418-0e5106915765","task_key":"default--validation--28985","task_revision_id":"1","upstream_id":"28985","short_description":"HellaSwag validation 28985","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do your hair like a dancer [title] Pull all your hair back into a tight ponytail, high on the back of your head. Secure with a hair elastic [title] Grasp your ponytail in your hand and begin to twist the hair to the left as you would ring out a towel. [title] Wrap your hair tightly in a counterclockwise circle around the ponytail holder. [step] Continue twisting until all hair is coiled.\",\"endings\":\"['[title] Tip the hair over to the right side of the ponytail holder. [step] Center the ponytail holder on the back of your head and twist the hair to the left.', 'Keep your hair tight and reverse this until all hair is twisted into the top of your head. [title] Twist your hair around until there is a gap in the ponytail.', 'You will now be covering the top of your ponytail with a hair tie. [title] You can now secure all of the loose bottom part of your hair.', '[title] Put another ponytail holder around the bun and the loose ends of the ponytail. [step] An optional method is to skip the second ponytail holder and pin as you go, adding pins approximately every 90 degrees.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}