{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7d7f51d-3bc3-5846-ae24-703cfbb11210","task_key":"default--test--7502","task_revision_id":"1","upstream_id":"7502","short_description":"If the market price is above the perfectly competitive firm's average total cost…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['there is no predictable change in the industry.', 'the industry stays the same as firms exit the market.', 'the industry contracts as no firms enter or exit the market.', 'the industry expands as firms exit the market.', 'the industry stays the same as firms enter the market.', 'the industry expands as firms enter the market.', 'the industry contracts as firms exit the market.', 'the industry contracts as firms enter the market.', 'the industry expands and contracts simultaneously as firms enter and exit.', 'the industry expands but no firms enter or exit the market.']\",\"question\":\"If the market price is above the perfectly competitive firm's average total cost curve, we expect that in the long run,\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}