{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62f3e244-3862-5ebb-90f2-f91d8a3ec087","task_key":"default--test--5564","task_revision_id":"1","upstream_id":"5564","short_description":"The statement of activities in the government-wide financial statements includes…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Separate columns for each major governmental fund.', 'Separate columns for each individual department in the government.', 'A single column combining all governmental activities.', 'A separate column for each governmental activity and business-type activity, but not for discretely presented component units.', 'A separate section at the top of the statement that shows program and general revenues.', 'Separate sections for governmental activities, business-type activities, but without discretely presented component units.', 'Separate columns for each major and nonmajor governmental fund.', 'Separate columns for governmental activities business-type activities and discretely presented component units.', 'Separate columns for each nonmajor governmental fund.', 'A separate section at the bottom of the statement that shows program and general revenues.']\",\"question\":\"The statement of activities in the government-wide financial statements includes which of the following?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}