{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1f082673-1998-5464-83b2-8ed38276ab2c","task_key":"all--test--1f082673-1998-5464-83b2-8ed38276ab2c","task_revision_id":"1","upstream_id":"","short_description":"One of the premises of the \"lack of imagination\" argument that Kamm discusses is…","config":"all","split":"test","body":"{\"choices\":\"['genetic enhancement is always morally permissible.', 'nature ought to be honored over personal choice.', \\\"most people's conception of the varieties of goods is limited.\\\", 'we ought to always leave the development of valuable traits up to chance.']\",\"question\":\" One of the premises of the \\\"lack of imagination\\\" argument that Kamm discusses is that\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}