{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2e0dc95-0580-5c19-8c68-925f595b0551","task_key":"default--validation--15604","task_revision_id":"1","upstream_id":"15604","short_description":"HellaSwag validation 15604","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wax ear hairs [title] Buy an at-home waxing kit. [step] Purchase a wax that is designed to be used on sensitive areas. You can use heated or non-heated wax.\",\"endings\":\"['Leave the kit in the same spot and wait three to six hours before using the product. [substeps] You can wax just one area of the ear.', 'Purchase an at-home waxing kit that is made to be used on sensitive skin. You can request wax directly from your dermatologist or pharmacist.', 'Non-heated wax removal products often come with the wax already on a strip, which can provide a more precise application of wax on ear hairs. This is the safest way to wax your ear hair.', 'A wax at home wax kit should contain waxing supplies, and work beside a heated wax tray in a cool, dark area. Your wax kit will also be used on affected areas, such as the backs of your nose and temple.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}