{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e84ab4d5-924e-5677-9b54-fc03007a0de0","task_key":"default--test--5047","task_revision_id":"1","upstream_id":"5047","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Congregational ministers.', 'Members of the American Anti-Slavery Society.', 'Supporters of the \\\"separate spheres\\\" ideology.', 'Southern supporters of the concept of \\\"female virtue.\\\"', 'Proponents of the \\\"cult of domesticity\\\" value system.', 'Supporters of the \\\"true womanhood\\\" ideal.', \\\"Opponents of the women's suffrage movement.\\\", 'Advocates for the patriarchal family structure.', 'Participants in the Seneca Falls Convention.', 'Advocates for the traditional gender roles in society.']\",\"question\":\"This question refers to the following information.\\n\\\"An act for the more effectual protection of the property of married women:\\n\\\"§1. The real property of any female who may hereafter marry, and which she shall own at the time of marriage, and the rents, issues, and profits thereof, shall not be subject to the sole disposal of her husband, nor be liable for his debts, and shall continue her sole and separate property, as if she were a single female.\\n\\\"§2. The real and personal property, and the rents, issues, and profits thereof, of any female now married, shall not be subject to the disposal of her husband; but shall be her sole and separate property, as if she were a single female, except so far as the same may be liable for the debts of her husband heretofore contracted.\\n\\\"§3. Any married female may take by inheritance, or by gift, grant, devise, or bequest, from any person other than her husband, and hold to her sole and separate use, and convey and devise real and personal property, and any interest or estate therein, and the rents, issues, and profits thereof, in the same manner and with like effect as if she were unmarried, and the same shall not be subject to the disposal of her husband nor be liable for his debts.\\\"\\n—Married Women's Property Act, New York State (1848)\\nWhich of the following groups would be most likely to support the perspective of the Married Women's Property Act?\",\"src\":\"ori_mmlu-high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}