{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b797b10f-747d-5317-9ba3-15f1c4eeb086","task_key":"default--test--713","task_revision_id":"1","upstream_id":"713","short_description":"Given the below information, what is the return on invested capital (to the…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['8.7%', '7.1%', '7.6%', '9.1%', '5.8%', '8.3%', '10.2%', '6.3%', '9.8%', '6.5%']\",\"question\":\"Given the below information, what is the return on invested capital (to the nearest tenth of 1%) ? Net investments (December 31, 1966) $250,000 Net investments (December 31, 1967) $350,000 Net sales year ended December 31, 1967 $600,000 Net income year ended December 31, 1967 $ 25,000\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}