{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b3a35ef-977f-523d-a715-9226bbb149bb","task_key":"default--test--2781","task_revision_id":"1","upstream_id":"2781","short_description":"A school psychologist is asked to help Ms. Smith address problem behaviors that…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Providing individual counseling to the 4 students who have not received discipline referrals', 'Assigning the 26 students who have received discipline referrals to in-school suspension.', 'Assigning detention to the 26 students who have received discipline referrals', 'Implementing a class-wide behavior management plan, such as a responsecost raffle.', 'Sending the 26 students who have received discipline referrals to an alternative school', 'Conducting a school-wide assembly to address the importance of proper behavior', 'Implementing a peer mediation program to address interpersonal conflicts', 'Providing school-based group counseling to address issues such as study skills, attendance, decision making, problem solving, and goal setting.', 'Using comprehensive emotional and behavioral assessments for students who have received discipline referrals', 'Developing individual behavior contracts for the 26 students who have received discipline referrals']\",\"question\":\"A school psychologist is asked to help Ms. Smith address problem behaviors that are disrupting the learning of her students. The school psychologist is given data that indicate discipline referrals have been written on 26 of the 30 students in Ms. Smith's class. Which of the following is best for the school psychologist to recommend?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}