{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4bd4e7a-9f1d-536c-97ea-6e43a3df3c83","task_key":"default--validation--23846","task_revision_id":"1","upstream_id":"23846","short_description":"HellaSwag validation 23846","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to diagnose dog flu [title] Look out for a cough. [step] Canine influenza is oftentimes confused with kennel cough, as both infections affect the respiratory tract. If your dog is coughing or showing other respiratory signs or difficulty breathing, you should bring him or her to the vet to be tested for (and possibly diagnosed with) canine influenza.\",\"endings\":\"['If your dog is all hard of hearing, has become sickly pale, is coughing regularly or is experiencing any of the above symptoms, take him to the vet immediately. [substeps] Kennel cough is often accompanied by an upset stomach and rapid breathing.', \\\"[title] Watch for unusual moisture and wetness around your dog's nose. [step] Your dog may have nasal discharge as a symptom of canine influenza.\\\", 'In most cases, however, the cough will respond to more critical screening options. [title] Note that influenza is not directly related to kennel cough.', 'Symptoms include: the dog gasping for air, coughing, pawing at his ears, and/or staggering through a collapsed chest. Your dog is shivering or having difficulty swallowing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}