{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86f20120-549f-570f-b8c1-93d1db3cd9dd","task_key":"default--test--7576","task_revision_id":"1","upstream_id":"7576","short_description":"The market demand curve for labor would shift to the left as the result of","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['a decrease in the marginal product of labor', 'an increase in the wage rate paid to workers', 'a decrease in the supply of labor', 'an increase in the supply of labor', 'a decrease in the wage rate paid to workers', 'an increase in demand for the good which the labor is producing', 'an increase in the price of the good which the labor is producing', 'an increase in the marginal product of labor']\",\"question\":\"The market demand curve for labor would shift to the left as the result of\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}