{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48f48043-c777-55d6-b5a8-ea05f2a400a4","task_key":"default--test--4436","task_revision_id":"1","upstream_id":"4436","short_description":"Infrared (IR) spectroscopy is useful for determining certain aspects of the…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['IR radiation can ionize organic molecules, providing structural information', 'absorption of IR radiation increases with molecular complexity', 'all molecular bonds absorb IR radiation', 'IR spectroscopy can identify isotopes within organic molecules', 'most organic functional groups absorb in a characteristic region of the IR spectrum', 'IR peak intensities are related to molecular mass', 'each element absorbs at a characteristic wavelength', 'IR spectroscopy can determine the polarity of organic molecules', 'the rate of IR absorption is proportional to the number of atoms in a molecule', 'all organic compounds reflect IR radiation']\",\"question\":\"Infrared (IR) spectroscopy is useful for determining certain aspects of the structure of organic molecules because\",\"src\":\"ori_mmlu-college_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}