{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27fd3bef-f125-53b2-af42-f7c2073d3a66","task_key":"default--test--7898","task_revision_id":"1","upstream_id":"7898","short_description":"A company that produces facial tissues continually monitors tissue strength. If…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Allowing the production process to continue when the tissue strength is within specifications.', 'Halting the production process without any data on tissue strength.', 'Halting the production process when the machinery needs inspection.', 'Not halting the production process despite customer complaints.', 'Allowing the production process to continue when the tissue strength is below specifications.', 'Halting the production process when the tissue strength is below specifications.', 'Allowing the production process to continue when the tissue strength is above specifications.', 'Halting the production process when sufficient customer complaints are received.', 'Halting the production process when the tissue strength is within specifications.', 'Halting the production process when the tissue strength is above specifications.']\",\"question\":\"A company that produces facial tissues continually monitors tissue strength. If the mean strength from sample data drops below a specified level, the production process is halted and the machinery inspected. Which of the following would result from a Type I error?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}