{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9b19579-8902-51d6-a661-a9d9e84824c8","task_key":"default--test--7054","task_revision_id":"1","upstream_id":"7054","short_description":"For a competitive firm, what is the most important thing to consider in deciding…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Do not produce if the TFC is not covered by revenue.', 'Do not produce if the price is lower than AVC.', 'Compare AVC to MR.', 'Compare AFC to MR.', 'Shut down if the firm is not making a profit.', 'Do not produce if the TVC is not covered by revenue.', 'Compare the price of inputs to the price of the product.', 'Compare TR to TC.', 'Produce the highest quantity demanded regardless of price.', 'Produce as long as the price is higher than AVC.']\",\"question\":\"For a competitive firm, what is the most important thing to consider in deciding whether to shut down in the short run?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}