{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f43eafcb-caf5-5f5f-a70d-c91e921c77a4","task_key":"all--test--f43eafcb-caf5-5f5f-a70d-c91e921c77a4","task_revision_id":"1","upstream_id":"","short_description":"For Socrates, the belief that \"virtue is knowledge\" is related to his claim that…","config":"all","split":"test","body":"{\"choices\":\"['the unexamined life is one in which we live day to day without asking questions about who we are and why we are here in the first place.', 'the Delphic oracle identified Socrates as the wisest person on earth because he claimed to know nothing.', 'by questioning traditional beliefs, we learn to recognize how some answers seem to be more satisfactory than others.', 'the only way to be a good or worthwhile person is to know how human beings should behave based on universal norms or values.']\",\"question\":\" For Socrates, the belief that \\\"virtue is knowledge\\\" is related to his claim that \\\"the unexamined life is not worth living,\\\" because he believes that:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}