{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27f2a3f1-9155-55a1-84cd-005849870fc7","task_key":"default--test--6906","task_revision_id":"1","upstream_id":"6906","short_description":"If the government wants to establish a socially optimal price for a natural…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the demand curve intersects the supply curve', 'marginal revenue equals zero', 'the average total cost equals zero', 'the average revenue equals the average total cost', 'the marginal cost curve intersects the demand curve', 'the marginal revenue equals the marginal cost', 'the marginal cost curve intersects the average total cost curve', 'the marginal revenue curve intersects the supply curve', 'the average total cost curve intersects the demand curve', 'average revenue equals zero']\",\"question\":\"If the government wants to establish a socially optimal price for a natural monopoly, it should select the price at which\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}