{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d3c25671-c19d-5b52-b7f3-8562c27d0de2","task_key":"default--test--6236","task_revision_id":"1","upstream_id":"6236","short_description":"An epidemic involving 10 individuals of all ages occurs in a migrant worker…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Epstein-Barr virus', 'Legionella pneumophila', 'Mycobacterium tuberculosis', 'Mycoplasma pneumoniae', 'Staphylococcus aureus', 'Streptococcus pyogenes', 'Corynebacterium diphtheriae', 'Streptococcus pneumoniae', 'Bordetella pertussis', 'Haemophilus influenzae']\",\"question\":\"An epidemic involving 10 individuals of all ages occurs in a migrant worker camp. None of the patients has had any form of immunization. Most cases present with fever, chills, tachypnea, sore throat, swollen glands, and severe malaise. One patient dies from respiratory complications, and another dies from myocardial involvement. A gram-positive rod is isolated from the respiratory tracts of 7 of the patients. The most likely cause of the outbreak is\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}