{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b1f1a78-3f84-56d5-933f-9c8f10efa9a5","task_key":"default--test--8441","task_revision_id":"1","upstream_id":"8441","short_description":"An experiment is conducted in which the response variable is the average gain in…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Taking more careful measurements to reduce variability in the response would increase the power of the test.', 'Change from equal size treatment groups to very different size treatment groups would increase the power of the test.', 'Increasing the sample size would reduce the probability of a Type I error.', 'Change from a 5% significance level to a 1% significance level would increase the power of the test.', 'Changing from a two-sample t-test to a one-sample t-test would increase the power of the test.', 'Reducing the size of the treatment groups would increase the power of the test.', 'Increasing the variability in the response would increase the power of the test.', 'Changing the response variable to the median gain in performance would increase the power of the test.', 'Decreasing the sample size would increase the power of the test.']\",\"question\":\"An experiment is conducted in which the response variable is the average gain in participants' performance in the long jump. A two-sample t-test with a 5% level of significance will be used to analyze the results. If all else is kept the same, which of the following descriptions of a possible change in procedure is true?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}