{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a8822b4-3f27-529b-9cf5-ed069a8003c5","task_key":"default--test--5716","task_revision_id":"1","upstream_id":"5716","short_description":"The city of Cobb has two trust funds for the benefit of the city's library trust…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Special revenue Special revenue', 'Permanent Capital projects', 'Capital projects Capital projects', 'Debt service Debt service', 'Permanent Permanent', 'Capital projects Permanent', 'Special revenue Permanent', 'Debt service Special revenue', 'Permanent Special revenue', 'Special revenue Debt service']\",\"question\":\"The city of Cobb has two trust funds for the benefit of the city's library trust fund A and trust fund B. Only the earnings from trust fund A can be expended and both the principal and interest from trust fund B can be expended. How should the city of Cobb report each trust fund? Trust fund A Trust fund B\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}