{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b529256-4cbf-591c-8d9a-56fc91ead57c","task_key":"all--test--7b529256-4cbf-591c-8d9a-56fc91ead57c","task_revision_id":"1","upstream_id":"","short_description":"The registrar’s office at a university has noticed that a large number of…","config":"all","split":"test","body":"{\"choices\":\"['The true proportion of students at this university with the correct address on record is within the confidence interval constructed.', 'Ninety-five percent of students at this university have the correct address on record.', 'The true proportion of students at this university with the correct addresses on record is within the confidence interval 95 percent of the time.', 'The sample proportion of students at this university with the correct address on record is within the confidence interval.']\",\"question\":\"The registrar’s office at a university has noticed that a large number of students fail to report a change of address. The registrar decides to take a random sample of 150 students from the current directory of students and determine the number of students with the correct addresses on record. He then uses this information to construct a 95 percent confidence interval. Which of the following statements must be true?\",\"subject\":\"high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}