{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7931dc26-0aa6-50cb-9320-f4871ff60119","task_key":"all--test--7931dc26-0aa6-50cb-9320-f4871ff60119","task_revision_id":"1","upstream_id":"","short_description":"For Plato, ordinary sensible objects exist and are knowable as examples or…","config":"all","split":"test","body":"{\"choices\":\"['in the sensible world only mathematical objects (e.g., triangles) can be known using hypotheses which are recollected when we are asked the right kinds of questions.', 'unlike everything in the sensible world, Forms are not individual things but rather the universal essences or natures by which individual things are what they are and are known.', 'nothing in the sensible, experienced world could exist or be identified as one particular thing or another unless there were a \\\"Sensible World\\\" Form (like the Form of beauty or justice).', 'the sensible world consists of changing Forms that exist and are known in terms of other changing Forms, which in turn exist and are known in terms of yet others in an endless regress.']\",\"question\":\"For Plato, ordinary sensible objects exist and are knowable as examples or instances of Ideas or \\\"Forms\\\" that do not exist in our ordinary sensible world.  Forms do not exist in the sensible world because:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}