{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c09e7aa-76bb-50ed-ae5d-22a19d96ba93","task_key":"default--test--7061","task_revision_id":"1","upstream_id":"7061","short_description":"Good X is exchanged in a competitive market. Which of the following is true if…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['If the demand curve is perfectly elastic, the price rises by the amount of the tax.', \\\"The consumer's burden of the tax rises as the demand curve is more elastic.\\\", 'Consumer surplus rises as a result of the tax.', \\\"The consumer's burden of the tax rises as the demand curve is less elastic.\\\"]\",\"question\":\"Good X is exchanged in a competitive market. Which of the following is true if an excise tax is now imposed on the production of good X?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}