{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2367b1c-679e-51e9-b16b-2f70420b8d6c","task_key":"default--test--8106","task_revision_id":"1","upstream_id":"8106","short_description":"Researchers are conducting an experiment using a significance level of 0.05. The…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The probability of a Type I error and the power would both increase.', 'The probability of a Type I error and the probability of a Type II error would both decrease.', 'The probability of a Type I error would stay the same and the power would increase.', 'The probability of a Type II error would decrease and the power would stay the same.', 'The probability of a Type I error would decrease and the probability of a Type II error would increase.', 'The probability of a Type II error and the power would both increase.', 'The power and the probability of a Type II error would both decrease.', 'The probability of a Type I error would increase and the power would decrease.', 'The power would stay the same and the probability of a Type II error would decrease.', 'The probability of a Type I error and the probability of a Type II error would both increase.']\",\"question\":\"Researchers are conducting an experiment using a significance level of 0.05. The null hypothesis is, in fact, false. If they modify their experiment to use twice as many experimental units for each treatment, which of the following would be true?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}