{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fcae4be-269e-57dd-a379-7c3aa19cbcb4","task_key":"all--test--1fcae4be-269e-57dd-a379-7c3aa19cbcb4","task_revision_id":"1","upstream_id":"","short_description":"Carens accepts that it is possible for the sudden opening of one country's…","config":"all","split":"test","body":"{\"choices\":\"['in practice, feasible policy options will not entail major costs to current citizens.', 'liberal egalitarianism is false, and so this possibility is irrelevant.', 'sometimes you have to do more harm than good in the short term to do more good in the long term.', 'all of the above']\",\"question\":\" Carens accepts that it is possible for the sudden opening of one country's borders to do more harm than good from a liberal egalitarian perspective. However, he argues that\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}