{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac095ffb-0015-558e-b084-355331b0d002","task_key":"default--test--2490","task_revision_id":"1","upstream_id":"2490","short_description":"based upon rewardor punishment or based upon a need to know. Distinguish on a…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Conformity, association, and adaptation', 'Agreement, alliance, and absorption', 'Compliance, identification, and internalization', 'Submission, emulation, and inculcation', 'Adherence, affiliation, and incorporation', 'Cooperation, imitation, and assimilation', 'Obedience, integration, and acculturation', 'Respect, imitation, and socialization', 'Acquiescence, imitation, and enculturation', 'Coordination, assimilation, and conformation']\",\"question\":\"based upon rewardor punishment or based upon a need to know. Distinguish on a more specific plane between the three kinds ofresponses to social influence.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}