{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48e275c2-7b51-5003-86bd-a05fed279e9e","task_key":"default--test--7514","task_revision_id":"1","upstream_id":"7514","short_description":"If a rich country and a poor country trade with each other, without any…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Exploitation cannot be determined solely based on monetary gains.', 'The concept of exploitation does not apply to international trade.', 'The rich country is not exploiting the poor country.', 'Both countries benefit equally in terms of percentage growth.', 'The trade terms are fair, and no exploitation occurs.', 'The poor country is benefiting more, considering the relative size of its economy.', 'The poor country is exploiting the rich country.', 'The gain is not significant enough to determine exploitation.', 'Both countries are exploiting each other.', 'The rich country is exploiting the poor country.']\",\"question\":\"If a rich country and a poor country trade with each other, without any restrictions, and the rich country gains $50 million, while the poor country only gains $5 million, is the rich country exploiting the poor country?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}