{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d7401f0b-19b8-5403-8c59-22693adcedc9","task_key":"all--test--d7401f0b-19b8-5403-8c59-22693adcedc9","task_revision_id":"1","upstream_id":"","short_description":"The Department of Health plans to test the lead level in a specific park.…","config":"all","split":"test","body":"{\"choices\":\"['Closing the park when the lead levels are within the allowed limit', 'Keeping the park open when the lead levels are in excess of the allowed limit', 'Closing the park when the lead levels are in excess of the allowed limit', 'Keeping the park open when the lead levels are within the allowed limit']\",\"question\":\"The Department of Health plans to test the lead level in a specific park. Because a high lead level is harmful to children, the park will be closed if the lead level exceeds the allowed limit. The department randomly selects several locations in the park, gets soil samples from those locations, and tests the samples for their lead levels. Which of the following decisions would result from the type I error?\",\"subject\":\"high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}