{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8db4ea69-e5ee-552e-9722-c1e567243bf8","task_key":"default--test--8978","task_revision_id":"1","upstream_id":"8978","short_description":"In general, how does tripling the sample size change the confidence interval…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['It multiplies the interval size by 9.', 'It does not change the interval size.', 'It doubles the interval size.', 'It triples the interval size.', 'It divides the interval size by 0.577.', 'It divides the interval size by 9.', 'It multiplies the interval size by 0.577.', 'It divides the interval size by 1.732.', 'It divides the interval size by 3.', 'It multiples the interval size by 1.732.']\",\"question\":\"In general, how does tripling the sample size change the confidence interval size?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}