{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"624c5a06-d9f5-5bc8-b2bf-75c65396dca3","task_key":"default--test--6237","task_revision_id":"1","upstream_id":"6237","short_description":"An epidemic involving 10 individuals of all ages occurs in a migrant worker…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Streptococcus pneumoniae', 'Mycoplasma pneumoniae', 'Staphylococcus aureus', 'Bordetella pertussis', 'Streptococcus pyogenes', 'Mycobacterium tuberculosis', 'Haemophilus influenzae', 'Legionella pneumophila', 'Corynebacterium diphtheriae', 'Epstein-Barr virus']\",\"question\":\"An epidemic involving 10 individuals of all ages occurs in a migrant worker camp. None of the patients has had any form of immunization. Most cases present with fever, chills, tachypnea, sore throat, swollen glands, and severe malaise. One patient dies from respiratory complications, and another dies from myocardial involvement. A gram-positive rod is isolated from the respiratory tracts of 7 of the patients. The most likely cause of the outbreak is \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}