{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0bd77e4-b538-55e8-9ede-286295a82119","task_key":"all--test--a0bd77e4-b538-55e8-9ede-286295a82119","task_revision_id":"1","upstream_id":"","short_description":"Socrates' claim that \"the unexamined life is not worth living\" is often cited as…","config":"all","split":"test","body":"{\"choices\":\"['it is sometimes simply not worth all the effort of examining life and its problems in great detail; sometimes it is better simply to \\\"go with the flow.\\\"', 'while taking a reflective attitude toward life is interesting and even sometimes important, most of what makes life worth living is not worth examining.', 'simply doing whatever everyone else does without thinking about why we should do what we do can hardly be thought of as worthwhile, noble, or admirable.', 'it is a waste of time to sit around thinking about whether life is worth living; we should leave such reflection to talk-show hosts, political figures, and religious leaders.']\",\"question\":\"Socrates' claim that \\\"the unexamined life is not worth living\\\" is often cited as a central theme in the activities of people.  By it, Socrates is typically understood to mean that:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}