{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be536ddb-4b30-5ee2-9eff-1a099323e1a2","task_key":"all--test--be536ddb-4b30-5ee2-9eff-1a099323e1a2","task_revision_id":"1","upstream_id":"","short_description":"Dr. Ja-Hoon Jang is conducting a research study that involves interviewing…","config":"all","split":"test","body":"{\"choices\":\"['tell the student that he cannot withdraw from the study since one of his parents has signed an informed consent.', 'tell the student that he cannot withdraw from the study since he (the student) agreed to participate at the beginning of the study.', 'contact the student’s parents to ask them to encourage their son to complete the interview.', 'allow the student to withdraw from the study.']\",\"question\":\"Dr. Ja-Hoon Jang is conducting a research study that involves interviewing junior high school students about their relationships with peers. Prior to beginning the study, Dr. Jang obtained informed consents from the parents of the students as well as assents from the students themselves. During the course of the study, one of the students says he doesn’t want to complete the interview. Dr. Jang should:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}