{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbcf8e81-166e-5139-a3e3-68dd6e3ed8ee","task_key":"default--validation--17592","task_revision_id":"1","upstream_id":"17592","short_description":"HellaSwag validation 17592","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to diagnose nocturnal asthma [title] Evaluate your coughing. [step] For many individuals with nocturnal asthma, coughing may be the only detectable symptom. If you believe you may have nocturnal asthma, it's important to assess how, when, and how intensely you cough.\",\"endings\":\"['Although pulling back the eyelids and covering your mouth with a plastic veil or scarf are common ways of determining your symptom, there are many others that effect the quality of your life. [substeps] Sometimes daytime coughing can lead to full-blown coughing.', '[substeps] Coughing typically occurs during early morning hours, especially between 2:00 am and 4:00 am. There is usually no mucus or phlegm being coughed up.', '[substeps] Excessive wheezing or crying is also a common symptom of nocturnal asthma. Wheezing is a symptom of asthma in infants, smaller children, and older adults.', 'At this point, you should visit a doctor to observe a history of your symptoms. [substeps] In some cases of asthma, coughing may be the only distinct symptom, though symptoms marked by coughing vary in species.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}