{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d8c9979-476b-5bac-9044-779c6d9be2c1","task_key":"all--test--1d8c9979-476b-5bac-9044-779c6d9be2c1","task_revision_id":"1","upstream_id":"","short_description":"According to the principle of greatest equal liberty proposed by John Rawls,…","config":"all","split":"test","body":"{\"choices\":\"['the highest level of happiness that is attainable by the least happy person in the society.', 'the social and economic inequalities that are built into society.', 'opportunities and resources that belong to that person but nobody else.', 'the most extensive basic liberty compatible with a similarly liberty for others.']\",\"question\":\" According to the principle of greatest equal liberty proposed by John Rawls, each person is to have an equal right to\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}