{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b32b69c-471e-551b-972d-4cb136a3f4f3","task_key":"default--test--7238","task_revision_id":"1","upstream_id":"7238","short_description":"If the government wishes to regulate a natural monopoly so that it produces an…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['where average total cost is equal to marginal cost.', 'where marginal revenue equals marginal cost.', 'where price is less than average total cost.', 'where price is less than marginal cost.', 'where price is equal to marginal cost.', 'where price is equal to average total cost.', 'where normal profits are made.', 'where average total cost is greater than marginal cost.', 'where price is greater than marginal cost.', 'where marginal revenue is greater than marginal cost.']\",\"question\":\"If the government wishes to regulate a natural monopoly so that it produces an allocatively efficient level of output, it would be at an output\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}