{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d88ea8cd-d69f-5304-b21e-8a7b23f51df2","task_key":"default--test--6682","task_revision_id":"1","upstream_id":"6682","short_description":"A 22-year-old male presents to the office with a 5-day history of diarrhea after…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Clostridium difficile', 'Entamoeba histolytica', 'Vibrio cholerae', 'Escherichia coli', 'Staphylococcus aureus', 'Campylobacter jejuni', 'Giardia lamblia', 'Salmonella typhi', 'Yersinia enterocolitica', 'Shigella dysenteriae']\",\"question\":\"A 22-year-old male presents to the office with a 5-day history of diarrhea after completing his third course of antibiotics for mastoiditis. Physical examination reveals vague generalized abdominal pain on palpation. Culture on hektoen enteric agar is positive. The most likely etiologic agent causing the diarrhea is\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}