{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfdf9925-d6ec-5de4-b3ac-108169a3960e","task_key":"all--test--dfdf9925-d6ec-5de4-b3ac-108169a3960e","task_revision_id":"1","upstream_id":"","short_description":"On the proposal that we need to establish world food banks to help those who are…","config":"all","split":"test","body":"{\"choices\":\"['if the proposal were to be realized, the operation must be conducted consistently.', 'only the richer countries have some moral obligation to make deposits in the world food banks.', 'it would be subject to the tragedy of the commons.', 'we need to go with the idea because we ought not to punish poor people who are caught in an emergency.']\",\"question\":\" On the proposal that we need to establish world food banks to help those who are in need, Hardin would say that\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}