{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65f0bc1e-fa63-56de-b40e-9f3a09fc5fb7","task_key":"default--test--6865","task_revision_id":"1","upstream_id":"6865","short_description":"What is the chief reason why farm policy was notconceived assimply part of an…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Geographic diversity of farming regions', 'Environmental concerns', 'Consumer preference for organic produce', 'Lack of funding', 'Technological advancements in agriculture', 'Public demand for food security', 'International trade agreements', 'Historical context', 'Influences from the agricultural industry', 'Political decisions']\",\"question\":\"What is the chief reason why farm policy was notconceived assimply part of an overall income distribution policy?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}