{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9aca636e-a00a-5a91-b0ec-11a5963e8395","task_key":"all--test--9aca636e-a00a-5a91-b0ec-11a5963e8395","task_revision_id":"1","upstream_id":"","short_description":"According to Socrates, the value or quality of one's life depends on…","config":"all","split":"test","body":"{\"choices\":\"['acting virtuously means acting in way that is informed about what one is doing and why.', 'someone who does not understand existence philosophically could never do anything right.', 'to have the power or ability to do anything at all requires that we know what we are doing.', 'not only is virtue knowledge but also the unexamined life is not worth living.']\",\"question\":\"According to Socrates, the value or quality of one's life depends on understanding the principles of, or basic rationale for human existence.  Without such knowledge (he suggests) life lacks virtue, because:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}