{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"862d21e1-11fd-5c36-8684-007c970f6497","task_key":"default--validation--35173","task_revision_id":"1","upstream_id":"35173","short_description":"HellaSwag validation 35173","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to handle poking wires on braces [title] Get some orthodontic wax. [step] Your orthodontist should have supplied you with some when you got your braces. [substeps] If you have run out, you can buy it at most pharmacies.\",\"endings\":\"['Orthodontic wax comes in small cases containing long strips of wax. If you cannot find wax at a pharmacy, call your orthodontist for some.', 'Wait for the wax to harden on your braces. Apply it slowly with a soft toothbrush to get wax off your teeth quickly.', \\\"If you usually have to use petroleum jelly, foam tape, or glitter, you don't have to do that. Just this tiny bit can make your brackets smoother and prevent painful twists.\\\", '[title] Assess whether you are allergic to your orthodontic wax. [step] If you have an allergic reaction to the wax, or if you have a very weak immune response, there is a good chance you have an allergy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}