{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90c86153-cce0-5d73-aa87-0ab31de1092d","task_key":"default--test--2525","task_revision_id":"1","upstream_id":"2525","short_description":"A school psychologist is providing feedback to a student's parents regarding the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the grade in which a student should be placed in school', 'the average score on that measure obtained by students in a given grade', \\\"the score that represents the student's potential ability in a particular grade level\\\", \\\"a measure of a student's academic progress compared to national standards\\\", \\\"the score that represents the student's current grade level performance regardless of age\\\", \\\"the score that represents the student's individual performance compared to other students\\\", 'the average score on that measure obtained by students at a given age', 'utilized to determine accountability among peers', \\\"used to determine a student's eligibility for special education services\\\"]\",\"question\":\"A school psychologist is providing feedback to a student's parents regarding the student's performance on a measure of academic achievement. To explain the concept of grade equivalent, the school psychologist should explain that it is\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}