{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c16481a9-2012-5c94-8afd-afb9330dd76b","task_key":"default--test--4161","task_revision_id":"1","upstream_id":"4161","short_description":"The photoelectron spectrum of carbon has three equally sized peaks. What peak is…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['The 2p peak has the lowest energy.', 'The 3d peak has the lowest energy.', 'The 2d peak has the lowest energy.', 'The 1d peak has the lowest energy.', 'The 1p peak has the lowest energy.', 'The 3s peak has the lowest energy.', 'The 4s peak has the lowest energy.', 'The 3p peak has the lowest energy.', 'The 2s peak has the lowest energy.', 'The 1s peak has the lowest energy.']\",\"question\":\"The photoelectron spectrum of carbon has three equally sized peaks. What peak is at the lowest energy?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}