{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45f6a53c-b913-527a-8c8f-3246028fb96f","task_key":"default--test--2780","task_revision_id":"1","upstream_id":"2780","short_description":"The superintendent of a large school district asks the school psychologist to…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The cost of administering the school interest inventory to all students', 'The correlation between the school interest inventory scores and the achievement test scores', 'The similarity between the school interest inventory scores and the aptitude test scores', \\\"The effect of the school interest inventory score on the students' second-year high school GPA\\\", \\\"The correlation between the school interest inventory score and the students' middle school GPA\\\", 'The degree to which the school interest inventory score raises the multiple correlation when it is included as a predictor variable of first-year high school GPA', 'The zero-order correlation of first-year high school GPA and the school interest inventory score', 'The difference in the correlations of the school interest inventory scores with first-year high school GPA and the aptitude and achievement test scores with first-year high school GPA', 'The time required to administer the school interest inventory', 'The intercorrelation of the aptitude and achievement test scores']\",\"question\":\"The superintendent of a large school district asks the school psychologist to predict students' grade point averages (GPAs) for the first year of high school. The school psychologist has included achievement and aptitude scores as predictor variables for firstyear high school GPA in a regression analysis and is considering whether to also administer a school interest inventory and include those scores. The major concern of the school psychologist when making this decision should be which of the following?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}