{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"125ac30e-55e3-5595-bb08-2361321264c1","task_key":"all--test--125ac30e-55e3-5595-bb08-2361321264c1","task_revision_id":"1","upstream_id":"","short_description":"The United States is trading salmon to Peru in exchange for anchovies. If these…","config":"all","split":"test","body":"{\"choices\":\"['The United States has comparative advantage in anchovy production, and Peru has comparative advantage in salmon production.', 'The United States has comparative advantage in salmon production, and Peru has comparative advantage in anchovy production.', 'The United States has absolute advantage in anchovy production, and Peru has absolute advantage in salmon production.', 'The United States has absolute advantage in salmon production, and Peru has absolute advantage in anchovy production.']\",\"question\":\"The United States is trading salmon to Peru in exchange for anchovies. If these nations are trading based upon relative opportunity costs, what must be the case?\",\"subject\":\"high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}