{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f69caeb-d0d9-55a4-87f0-731f9b12f3d9","task_key":"all--test--8f69caeb-d0d9-55a4-87f0-731f9b12f3d9","task_revision_id":"1","upstream_id":"","short_description":"Which of the following has provided the most information about the structure of…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Measurement of the intensity and fluctuations of Earth's magnetic field\\\", 'Examination of flowing lava', 'Collection of samples from deep boreholes drilled into Earth', 'Studies of the speeds and travel paths of seismic waves passing through Earth']\",\"question\":\"Which of the following has provided the most information about the structure of Earth's core, mantle, and lower crust?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}