{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4e828ab-da4f-5d06-95e2-57291d201bef","task_key":"default--test--6888","task_revision_id":"1","upstream_id":"6888","short_description":"Which is the best way to describe the AS curve in the long run?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Always upward sloping because it follows the Law of Supply.', 'Always downward sloping due to decreased demand over time.', 'Always upward sloping due to the effects of inflation.', 'Fluctuates between upward and downward sloping.', 'Always vertical in the short run.', 'Always downward sloping.', 'Always vertical in the long run.', 'Always horizontal.', 'Fluctuates between vertical and horizontal.', 'Always horizontal due to constant supply.']\",\"question\":\"Which is the best way to describe the AS curve in the long run?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}