{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6b94bb0-5dc4-54b5-b71a-28a438e04990","task_key":"default--test--7240","task_revision_id":"1","upstream_id":"7240","short_description":"If a business wants to increase its revenue and it knows that the demand price…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['maintain current price because demand is unit elastic', 'increase price because demand is inelastic', 'decrease price because demand is unit elastic', 'decrease price because demand is inelastic', 'increase price because demand is unit elastic', 'maintain current price because demand is inelastic', 'decrease price because demand is unit inelastic', 'decrease price because demand is elastic', 'maintain current price because demand is elastic', 'increase price because demand is elastic']\",\"question\":\"If a business wants to increase its revenue and it knows that the demand price elasticity of its product is equal to 0.78, it should\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}