{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74a6a861-e38a-577b-b8c8-f55cdd5bfdc1","task_key":"default--test--9356","task_revision_id":"1","upstream_id":"9356","short_description":"The terrestrial planet cores contain mostly metal because","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['the core is the only part of the planet that can withstand the high pressure needed to contain metals.', \\\"metals were drawn to the core by the planet's magnetic field.\\\", 'metals sank to the center during a time when the interiors were molten throughout.', 'metals are more resistant to heat and pressure, so they survived at the core while other materials melted away.', 'the entire planets are made mostly of metal.', 'radioactivity created metals in the core from the decay of uranium.', 'metals condensed first in the solar nebula and the rocks then accreted around them.', \\\"asteroids and comets delivered metals to the core during the early stages of the planet's formation.\\\"]\",\"question\":\"The terrestrial planet cores contain mostly metal because\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}