{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f23d21d-3ccd-5162-b588-817419e0cf2b","task_key":"default--test--7366","task_revision_id":"1","upstream_id":"7366","short_description":"When a monopolist has maximized profit,","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['any short-run profit will be eliminated through the long-run entry of new firms.', 'price is set equal to average total cost, creating positive economic profit.', 'output is set where price is equal to marginal revenue.', 'price is set above marginal cost, creating allocative inefficiency.', 'any short-run profit will be increased through the long-run exit of existing firms.', 'output is set where price is equal to average total cost.', 'output is set where price is less than average total cost.', 'price is set equal to average variable cost, creating zero economic profit.', 'price is set equal to marginal cost, creating zero economic profit.', 'price is set below marginal cost, creating economic inefficiency.']\",\"question\":\"When a monopolist has maximized profit,\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}