{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3166517f-f38b-583a-850c-d0345369ec80","task_key":"default--test--6279","task_revision_id":"1","upstream_id":"6279","short_description":"Which of the following is true?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Creatine supplements can increase muscle strength and power.', 'Creatine supplements have no effect on muscle strength and power.', 'Muscle growth is stimulated by a decrease in protein intake.', 'Increasing the protein intake above 3 grams per kg body mass per day will stimulate muscle growth and increase strength.', 'Increasing the protein intake above 3 grams per kg body mass per day will decrease muscle growth and strength.', 'Amino acid supplements can decrease muscle strength and power.', 'Protein supplements can decrease muscle strength and power.', 'Muscle damage is prevented by shortening contractions.', 'Amino acid supplements can increase muscle strength and power.', 'Muscle damage is induced by shortening contractions.']\",\"question\":\"Which of the following is true?\",\"src\":\"ori_mmlu-college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}