{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39b787a8-00a3-55bf-9aa2-4c82ef0f9c6d","task_key":"all--test--39b787a8-00a3-55bf-9aa2-4c82ef0f9c6d","task_revision_id":"1","upstream_id":"","short_description":"Carens's main conclusion is that","config":"all","split":"test","body":"{\"choices\":\"['liberal egalitarianism should almost always press for more openness toward immigrants and refugees.', 'liberal egalitarianism, contrary to appearances, entails that only closed border immigration policies are acceptable.', 'egalitarianism is founded on sentiments of envy and is therefore unjustified.', 'liberal egalitarian objections to anticosmopolitanism do not succeed.']\",\"question\":\" Carens's main conclusion is that\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}