{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"490053a4-4907-5094-a71b-c49646bd4283","task_key":"default--test--5255","task_revision_id":"1","upstream_id":"5255","short_description":"Which of the following is true of a profitmaximizing firm in the short run?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The firm will shut down if marginal revenue equals marginal cost but exceeds average variable cost.', 'The firm will continue to produce as long as total revenue exceeds total variable cost, even if it incurs loss.', 'The firm will shut down if average variable cost is greater than average total cost.', 'The firm will shut down if total revenue is equal to total variable cost.', 'The firm will shut down to minimize losses if product price falls below average total cost.', 'The firm will continue to produce as long as average fixed cost is less than average variable cost.', 'The firm will continue to produce as long as average variable cost is less than average fixed cost.', 'The firm will shut down if total variable cost exceeds total revenue.', 'The firm will continue to produce as long as total cost is less than total revenue.', 'The firm will continue to produce if average total cost is less than marginal cost.']\",\"question\":\"Which of the following is true of a profitmaximizing firm in the short run?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}