{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e244f356-3c60-5c1a-b2bd-fed33b1d5408","task_key":"all--test--e244f356-3c60-5c1a-b2bd-fed33b1d5408","task_revision_id":"1","upstream_id":"","short_description":"An unjust society, according to John Rawls, can be roughly characterized as any…","config":"all","split":"test","body":"{\"choices\":\"['some have more goods (e.g., happiness) than others.', 'political power is mostly given to the wealthy and upper class.', 'there are inequalities that are not to the benefit of all.', 'liberty is restricted to those who have not earned it.']\",\"question\":\" An unjust society, according to John Rawls, can be roughly characterized as any society where\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}