{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1edb95ab-0d31-5d0a-b4a4-3431bb394909","task_key":"default--test--7581","task_revision_id":"1","upstream_id":"7581","short_description":"Which of the following is a characteristic of monopolistic competition in the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Excess profits due to lack of competition.', 'Firms produce identical products.', 'High degree of interdependence among firms.', 'Zero economic profits.', 'Large number of buyers and one seller.', 'Minimum average total cost equals price.', 'Perfect knowledge of market conditions.', 'Homogeneous products.', 'Firms have total control over market price.', 'Strong barriers to entry.']\",\"question\":\"Which of the following is a characteristic of monopolistic competition in the long run?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}