{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4cc72a48-7179-5135-8920-f496b42ded45","task_key":"all--test--4cc72a48-7179-5135-8920-f496b42ded45","task_revision_id":"1","upstream_id":"","short_description":"A state generally provides funding for the medical care of its residents who…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the law reasonably conserves the state's limited resources.\\\", 'No, because the law reasonably prevents the expenditure of state funds on transient nonresidents.', \\\"Yes, because the law burdens the woman's fundamental right to health care.\\\", \\\"Yes, because the law burdens the woman's fundamental right to travel.\\\"]\",\"question\":\"A state generally provides funding for the medical care of its residents who cannot afford such care. State law, however, prohibits use of this state funding for surgery for any person who has resided in the state for less than one year, except in emergency situations. A woman moved to the state two months ago seeking permanent employment. Her physician recommends non-emergency surgery to treat a medical condition. The surgery would qualify for state funding if the woman had resided in the state for a year. The woman has sued to invalidate the state law that prohibits state funding of her surgery. Should the woman prevail in her action?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}