{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4183f7d-9fd3-548a-b7e5-97e6e1cf43dc","task_key":"default--test--2722","task_revision_id":"1","upstream_id":"2722","short_description":"The role of the outer ear is to","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['transduce sound waves to stimulate the ossicles', 'amplify sound waves before they reach the eardrum', 'conduct sound by exciting the cilia in the inner ear', 'control the balance and coordination of the body', 'produce earwax to protect the ear canal', 'collect and focus sounds from the air', 'regulate the pressure between the outer and middle ear', 'convert sound waves into electrical signals for the brain', 'filter out unnecessary noise before it reaches the inner ear', 'protect the eardrum while it transduces sound']\",\"question\":\"The role of the outer ear is to\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}