{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"361aa41e-306e-5558-bb0f-9be6ec461379","task_key":"default--test--9167","task_revision_id":"1","upstream_id":"9167","short_description":"How do astronomers think Jupiter generates its internal heat?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['the gravitational pull from its moons', 'heat produced by its magnetic field', 'by contracting changing gravitational potential energy into thermal energy', 'heat absorption from the Sun', 'nuclear fusion in the core', \\\"radioactive decay in the planet's core\\\", 'the burning of gases in its atmosphere', 'thermal radiation from its surface', 'internal friction due to its high rotation rate', 'chemical processes']\",\"question\":\"How do astronomers think Jupiter generates its internal heat?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}