{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"505c6e5d-dd1a-586e-b2aa-9d3e590bffba","task_key":"default--test--2566","task_revision_id":"1","upstream_id":"2566","short_description":"To decrease hostility between groups of junior high school students, you would…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Have the school counselor conduct individual sessions with each student.', 'assign the students goals that can be achieved only through intergroup cooperation.', 'have a respected member of the community speak to the students about the benefits of cooperation.', 'Teach students about the negative impacts of hostility through educational videos and presentations.', 'have each group select a representative to discuss ways to reduce hostility.', 'Implement a reward system for groups that show the least hostility.', 'Organize competitive games between the groups to foster healthy competition.', 'increase opportunities for contact under pleasant conditions.', 'Encourage each group to develop their own set of rules and guidelines.', 'Establish strict rules and penalties for hostile behavior.']\",\"question\":\"To decrease hostility between groups of junior high school students, you would be best advised to:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}