{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8ff653d-8350-5d2e-8e31-2549c81e024e","task_key":"default--test--6837","task_revision_id":"1","upstream_id":"6837","short_description":"Which of the following statements is true for a firm in a perfectly competitive…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Average revenue equals marginal revenue.', 'Marginal revenue is always increasing.', 'Average revenue is always negative.', 'Marginal revenue equals total revenue.', 'Average revenue is initially negative and then becomes positive.', 'Marginal revenue is decreasing.', 'Total revenue always decreases.', 'Total revenue increases and then decreases.', 'Total revenue equals average revenue.', 'Marginal revenue is initially positive and then becomes negative.']\",\"question\":\"Which of the following statements is true for a firm in a perfectly competitive industry?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}