{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7a0575d-3541-5908-991c-92369677c891","task_key":"default--test--7299","task_revision_id":"1","upstream_id":"7299","short_description":"A competitive labor market is currently in equilibrium. Which of the following…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The price of a substitute resource decreases.', 'The Department of Labor removes the need for workers to pass an exam before they can work in this field.', 'The government increases taxes on businesses in this labor market.', 'The price of a complementary resource increases.', 'The cost of living in the area significantly decreases.', 'A decrease in the supply of workers.', 'More students graduate with the necessary skills for this labor market.', 'Demand for the good produced by this labor is stronger.', 'The government introduces new labor laws that increase the cost of hiring.', 'The Department of Labor introduces a new certification requirement for workers in this field.']\",\"question\":\"A competitive labor market is currently in equilibrium. Which of the following most likely increases the market wage?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}