{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20f7d43d-b929-59c5-b0d8-103c2bd41deb","task_key":"default--test--687","task_revision_id":"1","upstream_id":"687","short_description":"Economic—marketing should encourage long-term economic development as opposed to…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Economic, emerging, and evolutionary', 'Evolutionary, experimental, and economic', 'Ecological, equitable, and economic', 'Equitable, evolutionary, and emerging', 'Evolutionary, equitable, and economic.', 'Economic, equitable, and experimental', 'Ecological, experimental, and economic', 'Ecological, equitable, and emerging.', 'Economic, ecological, and experimental', 'Ecological, evolutionary, and economic']\",\"question\":\" Economic—marketing should encourage long-term economic development as opposed to short-term economic development.\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}