{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27e79e02-ef73-5737-a4b6-0497939031ad","task_key":"default--test--2377","task_revision_id":"1","upstream_id":"2377","short_description":"After World War II there was much speculation about the personalities of those…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Nazi war criminals were forced to commit atrocities against their will', 'Nazi war criminals had unique and rare personality traits that predisposed them to commit such crimes', 'The personalities of Nazi war criminals were distinctly different from average citizens', 'The Milgram studies suggest that education level was the primary factor in predicting who would commit atrocities', 'Nazi war criminals were inherently evil', 'Nazi war criminals were exclusively psychopathic personalities', 'The Milgram studies show that obedience to authority does not significantly influence behavior', 'The atrocities were committed by a small group of Nazi war criminals with unique psychological disorders', 'Only those with a strong ideological alignment with Nazi beliefs committed atrocities', 'Individuals involved in the atrocities of the German concentration camps were likely of a personality structure not unlike the average citizen in America today']\",\"question\":\"After World War II there was much speculation about the personalities of those who had committed atrocities. What do the findings of Milgram's studies imply about the personalities of Nazi war criminals?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}