{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"37ac3eb3-3ec1-5f59-ad3a-1c2e7cd8b21c","task_key":"default--test--7324","task_revision_id":"1","upstream_id":"7324","short_description":"Suppose the market price for corn drops from $1,00per bushel to $.75 per bushel.…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The supply of corn to the market has increased.', 'No, the supply of corn to the market has not decreased.', 'Yes, the supply of corn to the market has decreased.', 'The supply of corn to the market remained the same.']\",\"question\":\"Suppose the market price for corn drops from $1,00per bushel to $.75 per bushel. In reaction to this, the farmers decide to sell one third less corn to the market. Has the supply of corn to the market decrease?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}