{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eaf4e25e-fd7a-568e-9ef9-b91566863304","task_key":"default--validation--25992","task_revision_id":"1","upstream_id":"25992","short_description":"HellaSwag validation 25992","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to apologize to a teen girl [title] Make a plan to meet up with her. [step] Apologies are best articulated when the parties involved are face to face. Apologizing in person means you set aside time out of your day specifically for this apology which will help her believe that you actually mean it.\",\"endings\":\"['[substeps] Arrive at the bar just a few minutes early for the perfect meeting time. Her mood will be better with just a few minutes to spare.', '[substeps] You could ask her to please meet you briefly at the local coffee shop or library. Or try asking if it would be okay for you to stop by her house so she can be in a comfortable environment for your apology.', \\\"[substeps] If you cannot get to see her in person, pick an appropriate time to talk to her. It might be a good time between 10 pm and 8 pm if there isn't too much down time in the forecast.\\\", \\\"[substeps] Don't text or meet up with her near the end of the night as it will be harder to study and she may become annoyed with you. You should make a phone plan to meet up for coffee about a half hour before class to catch up.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}