{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97edf83c-d655-5f10-8ca7-5cfc00a942f1","task_key":"default--test--7385","task_revision_id":"1","upstream_id":"7385","short_description":"One of the reasons Adam Smith supported the Free Market was because he believed…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Circular reasoning fallacy', 'Ad hominem fallacy', 'Slippery slope fallacy', 'Red herring fallacy', 'Hasty generalization', 'Straw man fallacy', 'Post hoc ergo propter hoc fallacy', 'Fallacy of composition', 'False dilemma fallacy', 'Appeal to ignorance fallacy']\",\"question\":\"One of the reasons Adam Smith supported the Free Market was because he believed that what is good for the individual is good for the country. So, since everyone would like to choose what they want to buy, it must be good for the country to give everyone free choice. Identify the error in Smith's logic and give a counter example.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}