{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7fa20258-13bd-54f5-84d4-14abc33fb14d","task_key":"default--test--1704","task_revision_id":"1","upstream_id":"1704","short_description":"A state has recently enacted a statute that provides no person or company may be…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The statute was overwhelmingly adopted by the voters of the state.', 'The state has the highest unemployment rate in the country.', \\\"The statute will increase the state's overall economic productivity.\\\", 'The statute will help racial minorities living in the state obtain gainful employment.', 'The statute will promote local businesses and stimulate economic growth within the state.', \\\"The statute will discourage out-of-state companies from taking advantage of the state's resources.\\\", 'If the state uses its own taxpayer funds to purchase construction materials, it is responsible for creating demand for the required labor.', 'The statute will ensure that the benefits of state contracts stay within the state.', 'The statute was enacted in response to a major construction company hiring out-of-state workers for a large project.', 'The statute will encourage more citizens to gain skills in the construction industry.']\",\"question\":\"A state has recently enacted a statute that provides no person or company may be awarded any state construction contract unless the person or company agrees to hire only citizens of the state. The primary purpose of the statute is to help alleviate the state's high rate of unemployment. Which of the following, if established, is the strongest argument in support of the statute if it is attacked as violating the commerce clause? \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}