{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9318fddf-50f9-5033-ba8a-87b61e7f71e1","task_key":"default--test--5540","task_revision_id":"1","upstream_id":"5540","short_description":"Of the following objectives, which is most appropriate for a technology…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['List the different types of machinery used in construction', 'Categorize the components of a technical system', 'Create a poster that details the process of building a bridge', 'Design and construct a threedimensional model of a low-income, multifamily dwelling unit', 'Explain the principles of architectural design', 'Identify the major tools and equipment used in highway construction', 'Memorize the names of various computer software programs', 'Write an essay on the history of technology education', 'Describe several construction careers that are related to home building', 'Discuss the environmental impact of deforestation for construction purposes']\",\"question\":\"Of the following objectives, which is most appropriate for a technology education program that has a goal of enabling students to develop creative technical solutions to present and future societal problems?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}