{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9132e038-9083-50ba-85ad-0ba42854e266","task_key":"default--test--5871","task_revision_id":"1","upstream_id":"5871","short_description":"Weber said that the 'spirit of capitalism' could be traced back to:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the movement towards religious pluralism', 'inspirational Protestant groups who revived religious ideas', 'the fall of feudalism and the rise of industrialism', 'the introduction of paper currency in the Middle Ages', 'new religious movements who rejected traditional forms of labour', 'the spread of secularism in modern society', 'Calvinists who engaged in ascetic practices to gain signs of salvation', 'the expansion of trade during the Renaissance', 'the development of the scientific method during the Enlightenment', 'the birth of democracy in ancient Greece']\",\"question\":\"Weber said that the 'spirit of capitalism' could be traced back to:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}