{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d65ed3d4-2135-5e5a-9677-9cddc5b66417","task_key":"all--test--d65ed3d4-2135-5e5a-9677-9cddc5b66417","task_revision_id":"1","upstream_id":"","short_description":"The terrestrial planet cores contain mostly metal because","config":"all","split":"test","body":"{\"choices\":\"['the entire planets are made mostly of metal.', 'radioactivity created metals in the core from the decay of uranium.', 'metals condensed first in the solar nebula and the rocks then accreted around them.', 'metals differentiated to the center during a time when the interiors were molten throughout.']\",\"question\":\"The terrestrial planet cores contain mostly metal because\",\"subject\":\"astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}