{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"608f48bd-0dc8-51b9-8d86-6ec112c2344f","task_key":"default--test--497","task_revision_id":"1","upstream_id":"497","short_description":"Replacing supply chains with _______ enhances the importance of product…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Supply loops, Reuse, Profit', 'Global production networks, Redesign, Profit', 'Supply loops, Recapture, Waste', 'Local production networks, Recapture, Waste', 'Local production networks, Redesign, Profit', 'Supply chains, Redesign, Profit', 'Global production networks, Reuse, Waste', 'Global production networks, Recapture, Waste', 'Supply loops, Redesign, Profit', 'Supply chains, Recapture, Waste']\",\"question\":\"Replacing supply chains with _______ enhances the importance of product _______as well as a fundamental redesign of every activity a firm engages in that produces _______.\",\"src\":\"ori_mmlu-business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}