{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f5cdd5c-c5b2-5921-a8c0-c578f04bbf6e","task_key":"default--test--5107","task_revision_id":"1","upstream_id":"5107","short_description":"Which of the following sequences of events is consistent with the presence of a…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['One-celled organisms developed a colony on the seafloor. Shells made by these organisms accumulated and lithified, forming the sandstone.', 'A meteorite impact created a crater. The impact pulverized rock into sand-sized particles that later lithified into sandstone.', 'A forest grew, died, and decayed on top of a layer of shale. The organic material transformed into sandstone over time.', 'The water level of a large lake lowered. A beach then formed where previously there had been lake bottom.', 'Mud was deposited and lithified. Subsequent contact metamorphism resulted in localized recrystallization of the shale into sandstone.', 'A volcano erupted, sending lava out over a layer of shale. The lava cooled and hardened into sandstone.', 'A river carried silt and clay that settled over a layer of shale. Over time, this material compacted into sandstone.', \\\"Tectonic activity pushed sandstone rock from deep within the Earth's surface to just above a layer of shale.\\\"]\",\"question\":\"Which of the following sequences of events is consistent with the presence of a layer of sandstone in bedrock just above a layer of shale?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}