{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff6a666a-c6d3-513f-bff7-c96ff8870657","task_key":"all--test--ff6a666a-c6d3-513f-bff7-c96ff8870657","task_revision_id":"1","upstream_id":"","short_description":"\"There is no rationale for myth because it is through myth that reason itself is…","config":"all","split":"test","body":"{\"choices\":\"['mythos is ultimately based on logos, just as myth is ultimately based on reasoning or thinking.', 'myth does not \\\"explain\\\" how things are related as much as it simply reveals them as related.', 'metaphysicians are justified in reasoning as they do because there is only one true answer about being.', 'myth and reason are the same: \\\"myth\\\" defines \\\"reason,\\\" and \\\"reason\\\" defines \\\"myth.\\\"']\",\"question\":\"\\\"There is no rationale for myth because it is through myth that reason itself is defined.\\\"  This means that:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}