{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb11be63-4990-5764-ba8f-474b2a1414e7","task_key":"default--validation--40881","task_revision_id":"1","upstream_id":"40881","short_description":"HellaSwag validation 40881","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to help a horse with choke [title] Look for common signs that a horse is choking. [step] One common sign is a lump in its neck. There are also several things that a horse will actually do when it is choking.\",\"endings\":\"['Despite showing these signs, be aware that a horse may still try to eat or drink. [substeps] Coughing yawning arching neck drooling difficulty breathing [title] Call the veterinarian immediately.', \\\"Watch out for : [substeps] Horses that choke often do not display any symptoms of choking. Typically, a horse won't do this behavior unless they have other, somewhat more serious medical issues.\\\", \\\"If the horse is vomiting, it will run straight for the side of the road. If the horse is having that bizarre coughing cough, it might pretend that there's no space in its stomach anymore, unless choking on acid or blood and wheezing a lot.\\\", 'When a horse has the choke, it will try to keep its mouth open to catch air. One of the most common signs of choke is a lump in its neck.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}