{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"07af73e2-3e85-538c-948c-41197cb59d68","task_key":"default--validation--38443","task_revision_id":"1","upstream_id":"38443","short_description":"HellaSwag validation 38443","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to give a neck massage [title] Place your partner in a comfortable seated position. [step] What's important is that his back can be comfortably straight. You must also be able to access his shoulders and upper back.\",\"endings\":\"[\\\"[title] Place your thumbs as close to the shoulder bones as possible. [step] If you still haven't reached those bones yet, place your thumbs anywhere around the side of the spine (between the chest and the upper back).\\\", \\\"[substeps] Before massaging your partner, pick a place where there is not a lot of moving material or noise around, such as the bottom of the tub. Massage one arm at a time and slowly work your way down your partner's body, working your way from his chest to the side or the back of his neck.\\\", '[substeps] Use a stool that gives you full access to his back. If you use a chair, make sure the back of the chair is low enough to give you access to the back of his shoulders.', 'Or better yet, the arms, legs, and chest. You may choose to have your hands resting on his shoulders, with your palms facing you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}