{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cfa08ac6-8c2c-5ebe-9f4b-da533f95ad77","task_key":"all--test--cfa08ac6-8c2c-5ebe-9f4b-da533f95ad77","task_revision_id":"1","upstream_id":"","short_description":"Dr. Bertha Black's advertisement for her seminar on relaxation techniques…","config":"all","split":"test","body":"{\"choices\":\"['ethical if she allows participants to leave after the lecture portion of the seminar.', 'ethical if the demonstration is well executed and educational.', 'unethical since she advertised the seminar as a “lecture.”', 'unethical because she plans to spend some of the time lecturing when the most beneficial part will probably be the experiential portion']\",\"question\":\"Dr. Bertha Black's advertisement for her seminar on relaxation techniques invites the public to “come hear an in-depth lecture on techniques to help reduce the stress of daily living.” The advertised cost for her four-hour seminar is $50. While preparing her lecture the night before the presentation, Dr. Black decides it would be valuable to demonstrate some of her techniques with the group. She decides that, instead of lecturing for the full four hours, she will have attendees divide into groups for the last two hours to experiment with the techniques. According to ethical standards, this is:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}