{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"140b64a9-2397-58df-a665-cf5cbb554de8","task_key":"default--test--8979","task_revision_id":"1","upstream_id":"8979","short_description":"A high school sent a survey to a randomly selected sample of 75 of last year's…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Use the surveys that were returned and conduct the analysis with a reduced confidence level.', 'Use the surveys that were returned and leave the sample size at 75 for the analysis.', 'Use the surveys that were returned and change the sample size to 48 for the analysis.', 'Randomly select another 75 class members and send the survey to them.', 'Send the survey to all members of the class to ensure a larger response rate.', 'Discard the survey results and opt for a different data collection method.', 'Re-send the survey to those that did not return it without further communication.', 'Use the surveys that were returned and estimate the responses of the 27 non-respondents.', 'Randomly select 27 additional class members and send the survey to them.', 'Follow up with those that did not return the survey to encourage them to respond.']\",\"question\":\"A high school sent a survey to a randomly selected sample of 75 of last year's graduating class. 27 of those selected did not return the survey. The best plan of action would be to:\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}