{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dedb2156-f8fa-5131-8232-5f720568079a","task_key":"default--test--8120","task_revision_id":"1","upstream_id":"8120","short_description":"The Department of Health plans to test the lead level in a specific park.…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Keeping the park open when the lead levels are within the allowed limit', 'Closing the park based on a predicted future lead level increase', 'Keeping the park open when the lead levels are in excess of the allowed limit', 'Closing other parks when their lead levels are within the allowed limit', 'Closing the park when the lead levels are in excess of the allowed limit', 'Not testing the lead levels at all and closing the park', 'Keeping other parks open when their lead levels are in excess of the allowed limit', 'Disregarding the test results and keeping the park open regardless of the lead levels', 'Closing the park when the lead levels are within the allowed limit', 'Closing the park based on lead levels from a different park']\",\"question\":\"The Department of Health plans to test the lead level in a specific park. Because a high lead level is harmful to children, the park will be closed if the lead level exceeds the allowed limit. The department randomly selects several locations in the park, gets soil samples from those locations, and tests the samples for their lead levels. Which of the following decisions would result from the type I error?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}