{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c2cd88e-46c9-5a28-b399-021f4618a682","task_key":"default--test--4667","task_revision_id":"1","upstream_id":"4667","short_description":"All the silicon and most of the oxygen in the earth's crust exists in the form…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Si^4+ : 1s^22s^22p^6, O^2- : 1s^22s^22p^6', 'Si^4+ : 1s^22s^22p^6, O^2- : 1s^22s^22p^4', 'Si^4+ : 1s^22s^22p^63s^1, O^2- : 1s^22s^22p^63s^23p^4', 'Si^4+ : 1s^22s^22p^63d^10, O^2- : 1s^22s^22p^63d^10', 'Si^4+ : 1s^22s^22p^63s^23p^2, O^2- : 1s^22s^22p^63s^23p^2', 'Si^4+ : 1s^22s^22p^63s^23p^6, O^2- : 1s^22s^22p^63s^23p^6', 'Si^4+ : 1s^22s^22p^63s^23p^2, O^2- : 1s^22s^22p^6', 'Si^4+ : 1s^22s^22p^63s^23p^4, O^2- : 1s^22s^22p^63s^1', 'Si^4+ : 1s^22s^22p^4, O^2- : 1s^22s^22p^63s^23p^6', 'Si^4+ : 1s^22s^1, O^2- : 1s^22s^22p^63s^2']\",\"question\":\"All the silicon and most of the oxygen in the earth's crust exists in the form of silicate minerals, such as granite. If the basic chemical unit of all silicates is the Sio_4^4- ion, which consists of an Si^4+ ion and four O^2- ions, write the electronic configurations of the Si^4+ and O^2- ions in silicate materials.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}