{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c50a4ae-c59b-575f-a873-fedda0fc7ee5","task_key":"default--test--5690","task_revision_id":"1","upstream_id":"5690","short_description":"Which of the following has provided the most information about the structure of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Studies of the gravitational pull of the Earth', 'Examination of flowing lava', \\\"Monitoring of Earth's tectonic plate movements\\\", 'Observations of volcanic eruptions', 'Inspection of surface rocks and minerals', 'Research on the composition of the atmosphere', 'Studies of the speeds and travel paths of seismic waves passing through Earth', 'Collection of samples from deep boreholes drilled into Earth', \\\"Measurement of the intensity and fluctuations of Earth's magnetic field\\\", 'Analysis of weather patterns and climate changes']\",\"question\":\"Which of the following has provided the most information about the structure of Earth's core, mantle, and lower crust?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}