{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7c46953-3b5c-5061-979d-1783d6aa1b07","task_key":"default--test--7440","task_revision_id":"1","upstream_id":"7440","short_description":"Relative to a competitive input market, a monopsony","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['pays less and hires more', 'pays the same and hires fewer', 'pays more and hires more', 'pays the same and hires more', 'pays more and hires the same amount', 'pays the same and hires the same amount', 'pays less and hires fewer', 'pays more and hires fewer', 'pays less and hires the same amount']\",\"question\":\"Relative to a competitive input market, a monopsony\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}