{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49ca92b7-f2b2-5a58-9c08-118b99bb0a2e","task_key":"default--test--6829","task_revision_id":"1","upstream_id":"6829","short_description":"If the world price of copper exceeds the domestic (U.S.) price of copper we…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the demand for U.S. copper to fall.', 'the United States to impose a tariff on imported copper to protect domestic producers.', 'the United States to subsidize domestic copper producers.', 'the United States to import more copper.', 'a decrease in the U.S. production of copper.', 'the demand for imported copper in the U.S. to fall.', 'a growing trade deficit in the United States in goods and services.', 'the domestic price of copper to increase.', 'the United States to be a net exporter of copper.', 'the United States to impose quotas on copper exports.']\",\"question\":\"If the world price of copper exceeds the domestic (U.S.) price of copper we would expect\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}