{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a060871e-00cb-56ba-8b42-99b3dcd073c0","task_key":"all--test--a060871e-00cb-56ba-8b42-99b3dcd073c0","task_revision_id":"1","upstream_id":"","short_description":"If the world that we individually perceive is limited to an internal…","config":"all","split":"test","body":"{\"choices\":\"['we know whether our internal perspective is correct only by comparing it with an objective, external perspective (the \\\"real\\\" world).', 'whatever we appeal to in order to prove that our perspective is right itself would be part of the standard we use in evaluating that perspective.', 'scientific research that reveals facts about the world would cause us to challenge our perceptions in a dreamworld of our own making.', 'without limiting our perspective to an internal dreamworld, we cannot achieve any objective, external knowledge of the real world.']\",\"question\":\"If the world that we individually perceive is limited to an internal perspective, then there is no way that we could determine whether our own perspective is useful, true, or valuable because:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}