{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76083ea3-aeff-5046-868c-27a2b6ab8f17","task_key":"all--test--76083ea3-aeff-5046-868c-27a2b6ab8f17","task_revision_id":"1","upstream_id":"","short_description":"In his discussion of the Divided Line, Plato says that, in contrast to mere…","config":"all","split":"test","body":"{\"choices\":\"['to what our senses reveal to us about how things appear to us, not how they  really are.', 'beyond the Forms to images of goodness, beauty, and truth obtained from particular objects.', 'to what we sincerely believe is true about the Forms based on our experiences in the world.', 'beyond sense experience to unchanging ideas (Forms) that are perceived as rationally ordered.']\",\"question\":\" In his discussion of the Divided Line, Plato says that, in contrast to mere belief or opinion, knowledge is a belief for which we give reasons or justifications by appealing:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}