{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee3fbb24-cbfa-5b29-b121-92c15cb26c5d","task_key":"all--test--ee3fbb24-cbfa-5b29-b121-92c15cb26c5d","task_revision_id":"1","upstream_id":"","short_description":"If it were known that a policy would wipe out several animal species without…","config":"all","split":"test","body":"{\"choices\":\"['this policy is morally problematic because it harms the environment.', 'this policy is morally problematic because it damages the integrity of the ecosystem.', 'this policy is morally obligatory because the environment ought not to exist.', 'this policy is morally unproblematic.']\",\"question\":\" If it were known that a policy would wipe out several animal species without negatively affecting human beings, Baxter would most likely say that\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}