{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22dc092e-4ddf-5c65-b155-6959e501e86b","task_key":"default--test--7053","task_revision_id":"1","upstream_id":"7053","short_description":"If the government subsidizes the production of halogen headlights,","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the price of halogen headlights will increase.', 'the supply curve will shift to the left.', 'the price of halogen headlights will remain the same.', 'the quantity of halogen headlights demanded will decrease.', 'both the supply and demand curves will shift to the left.', 'both the supply and demand curves will shift to the right.', 'the supply curve will shift to the right.', 'the demand curve will shift to the left.', 'there will be no change in the market for halogen headlights.', 'the demand curve will shift to the right.']\",\"question\":\"If the government subsidizes the production of halogen headlights,\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}