{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a869a01-cd82-5e5f-9d3b-b17278008a78","task_key":"all--test--3a869a01-cd82-5e5f-9d3b-b17278008a78","task_revision_id":"1","upstream_id":"","short_description":"A company created a new brand of pies. However, a study by the federal Food and…","config":"all","split":"test","body":"{\"choices\":\"[\\\"constitutional, because it is within the state's police power.\\\", 'constitutional, because Congress may not regulate an economic activity where both buyer and seller reside in the same state.', 'unconstitutional, because it affects interstate commerce.', 'unconstitutional, because the federal law preempts any conflicting state legislation regarding the sale and shipment of the pies.']\",\"question\":\"A company created a new brand of pies. However, a study by the federal Food and Drug Administration revealed that the pies contain potentially harmful levels of nuts for some consumers with nut allergies. As a result, Congress enacted legislation prohibiting the shipment and sale of the pies across state lines. A state has a statute that regulates the shipment and sale of the pies within its territory. In light of the federal legislation prohibiting the shipment and sale of the pies across state lines, the state statute is probably\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}