{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f76cc43f-1676-553f-a8e0-942370a539fd","task_key":"default--test--1810","task_revision_id":"1","upstream_id":"1810","short_description":"Congress enacts a $100 tax on the sale of any handgun to a private individual…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, if the tax is applied uniformly across all states.', 'Yes, if Congress could have banned possession of handguns outright.', 'No, if the tax is seen as a way to indirectly ban handguns.', 'Yes, if the tax is used to fund law enforcement or military activities.', 'No, if the tax disproportionately affects individuals of lower income.', \\\"No, because the tax infringes on an individual's right to bear arms.\\\", 'No, if the tax does not result in a significant collection of revenue.', 'No, because the tax is clearly intended as a penalty on handgun ownership.', 'Yes, if the dominant intent of Congress was that the tax would produce revenue.', 'Yes, if the tax is proportional to the cost of the handgun.']\",\"question\":\"Congress enacts a $100 tax on the sale of any handgun to a private individual not for use in law enforcement or military duties. Will this new handgun tax survive a constitutional challenge?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}