{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fdb6aaf9-91ea-5502-85e6-049b422b353b","task_key":"default--test--6249","task_revision_id":"1","upstream_id":"6249","short_description":"A 28-year-old man comes to the office because he would like to quit smoking…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Hypoglycemia', 'Heart arrhythmia', 'Renal insufficiency', 'Respiratory distress', 'Increased anxiety', 'Seizures', 'Liver toxicity', 'The patient is at no increased risk', 'Hypertension', 'Polyuria']\",\"question\":\"A 28-year-old man comes to the office because he would like to quit smoking cigarettes. He says, \\\"I cannot go more than a day without cigarettes because I get so anxious and irritable. I even wake up at night to smoke.\\\" The patient began smoking cigarettes at age 12 years; he was smoking two packs of cigarettes daily by age 17 years, and he has not curtailed his use since that time. He does not drink alcoholic beverages or use illicit drugs. Medical history is remarkable for childhood febrile seizures. He takes no medications. Vital signs are normal. The patient is interested in using a nicotine patch and bupropion therapy because he says his friend found the treatments to be helpful. In combining these two therapies, this patient is at greatest risk for which of the following? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}