{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf291138-7133-505a-9566-0242d872558d","task_key":"default--test--9900","task_revision_id":"1","upstream_id":"9900","short_description":"The entire power from a 100-hp automobile engine is used to agitate 50 kg of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['15 s', '60 s', '28 s', '10 s', '35 s', '40 s', '20 s', '25 s', '45 s', '50 s']\",\"question\":\"The entire power from a 100-hp automobile engine is used to agitate 50 kg of water thermally insulated from its surroundings. How long will it take for the temperature of the water to rise 10 Celsius degrees?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}