{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bb0f505b-a12e-5e96-a2ab-e7a7dce35145","task_key":"default--test--7130","task_revision_id":"1","upstream_id":"7130","short_description":"If every country in the world has the exact same opportunity costs, so that no…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Trade would cease without a comparative advantage', 'Trade is solely based on comparative advantage', 'Differences in technology and production methods could create opportunities for trade, even if the opportunity costs are the same.', 'Without comparative advantage, it is impossible for trade to occur', 'Differences in taste, leading to exchange between nations, would stimulate trade regardless of the absence of comparative advantages.']\",\"question\":\"If every country in the world has the exact same opportunity costs, so that no nation has a comparative advantage in any line of production, could there still be a basis for trade?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}