{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"169dea8a-09c2-5b76-b6a9-ae69a17e3fab","task_key":"default--test--8337","task_revision_id":"1","upstream_id":"8337","short_description":"Suppose there is a correlation of r = 0.9 between number of hours per day…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['90% of the variation in number of study hours per day can be explained by variation in GPAs.', 'If a student studies less, their GPA will always decrease.', '90% of students who study receive high grades.', '81% of the variation in GPAs can be explained by variation in number of study hours per day.', 'The correlation between number of hours studied per day and GPA is not significant.', '81% of students who study receive high grades.', '90% of students who study a lot receive low grades.', '90% of the variation in GPAs can be explained by variation in number of study hours per day.', 'If a student studies more, their GPA will always increase.', '90% of students who receive high grades study a lot.']\",\"question\":\"Suppose there is a correlation of r = 0.9 between number of hours per day students study and GPAs. Which of the following is a reasonable conclusion?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}