{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"620aac82-0f7d-5dba-a542-b5d554700bfa","task_key":"all--test--620aac82-0f7d-5dba-a542-b5d554700bfa","task_revision_id":"1","upstream_id":"","short_description":"Plato distinguishes knowledge from mere belief or opinion by saying that…","config":"all","split":"test","body":"{\"choices\":\"['grounded ultimately in what our senses reveal to us about the world of becoming.', 'based on images of the good, beauty, and truth obtained from particular objects and on which the concepts and Forms depend.', 'replaced by what we sincerely believe is true or have come to believe based on our upbringing.', 'understood in terms of concepts or innate ideas (Forms) that are perceived as rationally ordered.']\",\"question\":\"Plato distinguishes knowledge from mere belief or opinion by saying that knowledge must be a true belief for which one can give a justification, a rationale, or \\\"logos.\\\"  In terms of his image of the Divided Line, for Plato, knowledge is attained only when our sensible experience is:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}