{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e7fcc44-d7d4-5db7-8f8d-7c3b4c674203","task_key":"all--test--6e7fcc44-d7d4-5db7-8f8d-7c3b4c674203","task_revision_id":"1","upstream_id":"","short_description":"A state has recently enacted a statute that provides no person or company may be…","config":"all","split":"test","body":"{\"choices\":\"['The statute will help racial minorities living in the state obtain gainful employment.', 'The state has the highest unemployment rate in the country.', 'If the state uses its own taxpayer funds to purchase construction materials, it is responsible for creating demand for the required labor.', 'The statute was overwhelmingly adopted by the voters of the state.']\",\"question\":\"A state has recently enacted a statute that provides no person or company may be awarded any state construction contract unless the person or company agrees to hire only citizens of the state. The primary purpose of the statute is to help alleviate the state's high rate of unemployment. Which of the following, if established, is the strongest argument in support of the statute if it is attacked as violating the commerce clause?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}