{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d3929faf-151e-5791-986d-52f4246b7b86","task_key":"default--test--8007","task_revision_id":"1","upstream_id":"8007","short_description":"A group of scientists wanted to estimate the proportion of geese returning to…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The shape of the distribution of the sample proportion will change significantly.', 'There will be no noticeable change in the distribution of the sample proportion.', 'The distribution of the sample proportion will become more uniform.', 'The distribution of the sample proportion will become skewed.', 'The distribution of the sample proportion will be more spread out.', 'The distribution of the sample proportion will more closely resemble the binomial distribution.', 'The distribution of the sample proportion will be less spread out.', 'The spread of the distribution of the sample proportion will remain unaffected.']\",\"question\":\"A group of scientists wanted to estimate the proportion of geese returning to the same site for the next breeding season. Suppose they decided to increase the sample size from 200 to 2,000. How will this affect the distribution of the sample proportion?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}