{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c759c65-cd28-5d9f-846a-787123600ed5","task_key":"default--test--4925","task_revision_id":"1","upstream_id":"4925","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The spread of the Black Death in Europe', 'The discovery of the New World by Christopher Columbus', 'The conquest of India by rival Muslim empires', 'The rise of absolute monarchies in Europe', 'The invention of the printing press', 'The Scientific Revolution', 'The importation of food crops from the New World', 'The Protestant Reformation', 'The African slave trade', 'The development of joint-stock companies']\",\"question\":\"This question refers to the following information.\\n\\\"MAHATMA GANDHI'S MESSAGE.\\nREMAIN NON-VIOLENT\\nThe following message was sent by Mahatma Gandhi from Waltair:—\\nMaulana Mohamed AH was arrested at Waltair under sections 107 and 108 to be called upon to give security, to be of good behaviour for one year. The place and date of trial is unknown.\\nThe Begum Saheba and Mr. Hayat were permitted to see him after arrest.\\nHe and I were going to address a meeting outside the station. He was arrested. I continued going to the meeting and addressed them. There is no cause for sorrow, but every cause for congratulation. There should be no hartal. Perfect peace and calmness should be observed. I regard the arrest as a prelude to Swaraj and the redress of the Khilafat and the Punjab wrongs, if we can remain non-violent. Retain Hindu-Muslim Unity despite the madness of some Moplahs, and fulfil the Swadeshi programme.\\nI hope every Indian, man or woman, will completely boycott foreign cloth and take up spinning or weaving during every spare minute.\\nBy striving like the Maulana, be insistent on religious and national rights.\\nLet us earn imprisonment. I am conscious of the Maulana's innocence and I am sure the imprisonment of the innocent will enable the nation to reach the cherished goal.\\\"\\nMohandas Gandhi, The Pilgrims' March, 1921\\nWhich of the following historical developments from the period 1450–1750 C.E. most inspired the instructions indicated in the fifth paragraph to boycott foreign cloth?\",\"src\":\"ori_mmlu-high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}