{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"536c158e-e1b6-5fd7-9ee4-64a3e9a20c64","task_key":"default--test--4163","task_revision_id":"1","upstream_id":"4163","short_description":"Cerium(III) sulfate, Ce2(SO4)2, is less soluble in hot water than it is in cold.…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['The hydration energies of cerium ions and sulfate ions are very high.', 'The heat of solution of cerium(III) sulfate is isothermic.', 'The solubility of cerium(III) sulfate increases with temperature.', 'The heat of solution of cerium(III) sulfate is exothermic.', 'The hydration energies of cerium ions and sulfate ions are very low.', 'The solution is not an ideal solution.', 'The solubility of cerium(III) sulfate is independent of temperature.', 'The heat of solution for cerium(III) sulfate is endothermic.', 'The solution of cerium(III) sulfate is a supersaturated solution.', 'The heat of solution of cerium(III) sulfate has no effect on its solubility.']\",\"question\":\"Cerium(III) sulfate, Ce2(SO4)2, is less soluble in hot water than it is in cold. Which of the following conclusions may be related to this?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}