{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6bdff171-33f9-51e4-816b-e212ad7ec697","task_key":"default--test--7601","task_revision_id":"1","upstream_id":"7601","short_description":"How might the traditional view of economic efficiency in an oligopolyand the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Historical data contradicts theory', 'Oligopolies are less common in practice than in theory', 'There is no contradiction between theory and practice', 'Historical data is insufficient to determine the efficiency of oligopolies', 'Oligopolies typically demonstrate higher innovation rates than predicted', 'Oligopolies always operate inefficiently', 'Theory suggests oligopolies lead to perfect competition in the long run', 'Oligopolies always result in lower prices for consumers', 'Historical data supports theory']\",\"question\":\"How might the traditional view of economic efficiency in an oligopolyand the historical facts surrounding many oligopol-isticmanufacturing industries disagree with each other?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}