{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"caeb1a6c-938d-55ad-89c0-9b08f6a5706c","task_key":"default--test--8781","task_revision_id":"1","upstream_id":"8781","short_description":"A college admissions officer is interested in comparing the SAT math scores of…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The population of SAT scores from each group is not normally distributed.', 'The SAT scores from each group are unrelated.', 'The mean SAT scores from each group are similar.', 'The SAT scores from each group are skewed.', 'The population variances from each group are known.', 'The population of SAT scores from each group is normally distributed.', 'The population variances from the two groups are unequal.', 'The population variances from each group are unknown.', 'The population variances from the two groups are equal.', 'The population variances from each group are estimated.']\",\"question\":\"A college admissions officer is interested in comparing the SAT math scores of high school applicants who have and have not taken AP Statistics. She randomly pulls the files of five applicants who took AP Statistics and five applicants who did not, and proceeds to run a t-test to compare the mean SAT math scores of the two groups. Which of the following is a necessary assumption?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}