{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2d61f1c-6832-5cee-93d6-21dc5e55e6c4","task_key":"default--test--4254","task_revision_id":"1","upstream_id":"4254","short_description":"Which of the following statements most accurately explains why the 1H spectrum…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Both 35Cl and 37Cl have electric quadrupole moments.', 'The molecule is not rigid.', 'The hydrogen atom undergoes slow intermolecular exchange.', 'The molecule is in constant rotation.', 'Both 35Cl and 37Cl have I = 0.', 'The hydrogen atom undergoes rapid intermolecular exchange.', 'The molecule is rigid.', 'The hydrogen atom has a magnetic dipole moment.', 'Both 35Cl and 37Cl have no electric quadrupole moments.', 'Both 35Cl and 37Cl have magnetic dipole moments.']\",\"question\":\"Which of the following statements most accurately explains why the 1H spectrum of 12CHCl3 is a singlet?\",\"src\":\"ori_mmlu-college_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}