{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bbd7cd3-f684-5e9f-987a-3295fccf10d4","task_key":"all--test--1bbd7cd3-f684-5e9f-987a-3295fccf10d4","task_revision_id":"1","upstream_id":"","short_description":"Researchers are conducting an experiment using a significance level of 0.05. The…","config":"all","split":"test","body":"{\"choices\":\"['The probability of a Type I error and the probability of a Type II error would both decrease.', 'The probability of a Type I error and the power would both increase.', 'The probability of a Type II error and the power would both increase.', 'The probability of a Type I error would stay the same and the power would increase.']\",\"question\":\"Researchers are conducting an experiment using a significance level of 0.05. The null hypothesis is, in fact, false. If they modify their experiment to use twice as many experimental units for each treatment, which of the following would be true?\",\"subject\":\"high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}