{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b8a4f30-de31-5338-9a7a-9d8d6edc2388","task_key":"default--validation--32342","task_revision_id":"1","upstream_id":"32342","short_description":"HellaSwag validation 32342","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to determine if you have pneumonia [title] Identify symptoms of pneumonia. [step] If you think you might have pneumonia, it is important to treat it promptly before it gets worse. The symptoms may worsen gradually over several days or suddenly be very severe from the onset.\",\"endings\":\"['If you have ever had pneumonia, call your doctor to make an appointment for evaluation and exam if needed. [substeps] Warning signs of pneumonia include: tingling or tightness in your chest or abdomen, difficulty breathing, severe cramping, diarrhea, and over night sweats.', \\\"Pneumonia is very difficult to treat, but following the precautions shown in the steps above can help with the trial and error of a more accurate diagnosis. [substeps] It is a good idea to get a doctor's log detailing symptoms of pneumonia.\\\", 'Signs of pneumonia include : [substeps] Fever sweating and shivering discomfort in your chest when you cough or breathe, particularly breathing when breathing deeply fast, shallow breathing. This may occur only when you are physically active.', \\\"If you have pneumonia, take your doctor's exam or even a blood test to rule out pneumonia. [substeps] Pneumonia is a mycoplasma type.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}