{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58e67278-acfd-5e0f-a75e-b3c558acc367","task_key":"default--test--8123","task_revision_id":"1","upstream_id":"8123","short_description":"For which of the following is it appropriate to use a census?","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['A two-tailed hypothesis test where the null hypothesis was that the mean expenditure on entertainment by male students at a high school is the same as that of female students.', 'A 95% confidence interval of the proportion of students in a small town who are taking some AP class.', 'A 99% confidence interval of the mean weight of all students in a large college.', 'A confidence interval of the mean income of all families in a country.', 'A one-tailed hypothesis test where the null hypothesis is that the proportion of men over 40 in a city who exercise regularly is greater than 50%.', 'A 95% confidence interval of mean height of teachers in a small town.', 'A confidence interval of the mean age of all employees in a large corporation.', 'A 95% confidence interval of the proportion of people in a large city who have a pet.', 'A two-tailed hypothesis test where the null hypothesis is that the mean height of all basketball players in a country is the same as that of soccer players.', 'None of the above.']\",\"question\":\"For which of the following is it appropriate to use a census?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}