{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc2b6c20-08e9-5223-8c99-7bcbe996926c","task_key":"default--test--8343","task_revision_id":"1","upstream_id":"8343","short_description":"The registrar’s office at a university has noticed that a large number of…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The sample proportion of students at this university with the correct address on record is within the confidence interval.', 'Ninety-five percent of the students in the sample have the correct address on record.', 'The confidence interval is accurate for all students, not just the sample.', 'The true proportion of students at this university with the correct address on record is outside the confidence interval 95 percent of the time.', 'The sample proportion of students at this university with the incorrect address on record is within the confidence interval.', 'The true proportion of students at this university with the correct address on record is within the confidence interval constructed.', 'The true proportion of students at this university with the correct addresses on record is within the confidence interval 95 percent of the time.', 'Ninety-five percent of students at this university have the correct address on record.', 'The true proportion of students at this university with incorrect addresses on record is within the confidence interval.', 'The sample size of 150 students is within the confidence interval.']\",\"question\":\"The registrar’s office at a university has noticed that a large number of students fail to report a change of address. The registrar decides to take a random sample of 150 students from the current directory of students and determine the number of students with the correct addresses on record. He then uses this information to construct a 95 percent confidence interval. Which of the following statements must be true?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}