{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d2478cb-8e63-55fc-9b41-24e6ae52a9d3","task_key":"default--test--7191","task_revision_id":"1","upstream_id":"7191","short_description":"How did Joseph Schumpeter distinguish the innovatorfrom the inventor? On which…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Innovators invest in inventions and introduce them into industry', 'Inventors create new products, while innovators improve existing ones', 'Inventors invest in new inventions', 'Innovators replicate existing inventions without modification', 'Innovators make new discoveries', 'Innovators primarily focus on scientific research', \\\"Innovators don't take financial risks\\\", 'Innovators are solely responsible for funding inventions', 'Inventors are the ones who commercialize new products', 'Inventors develop new ideas, while innovators are responsible for marketing']\",\"question\":\"How did Joseph Schumpeter distinguish the innovatorfrom the inventor? On which did he base his theory of the business cycle?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}