{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"940a27e0-8e9f-51fb-964c-c585ce97518a","task_key":"all--test--940a27e0-8e9f-51fb-964c-c585ce97518a","task_revision_id":"1","upstream_id":"","short_description":"Why is it \"easier said than done\" to ensure that the principle of informed…","config":"all","split":"test","body":"{\"choices\":\"['It is not practicable to present every participant with all the information about the study.', 'Sometimes it is desirable to withhold certain pieces of information, such as the length of time an interview will take.', 'If the participants knew exactly what the researcher was intending to study, they might change their behavior.', 'All of the above.']\",\"question\":\"Why is it \\\"easier said than done\\\" to ensure that the principle of informed consent is adhered to?\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}