{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"161024c2-9b5f-52f5-9965-38f7cfc27d49","task_key":"all--test--161024c2-9b5f-52f5-9965-38f7cfc27d49","task_revision_id":"1","upstream_id":"","short_description":"During spring break, a private boarding school was deserted while students and…","config":"all","split":"test","body":"{\"choices\":\"['granted, because the police should have secured a warrant before seizing the envelope.', 'granted, because the police should have secured a warrant before opening the envelope.', 'denied, because the search was conducted by a private party.', 'denied, because the counselor, as a school employee, was in loco parentis.']\",\"question\":\"During spring break, a private boarding school was deserted while students and teachers were away on vacation. A guidance counselor remained on campus because he was working on a research project. After working late one night, the counselor decided to enter the room of a student from a very wealthy family. The counselor was rummaging through the student's room looking for something valuable to steal. Under the bed, he noticed an expensive suitcase. The counselor opened the suitcase and found an express mail envelope. The counselor knew that the student's father often sent money to his son in express mail envelopes. The counselor opened the envelope and saw that it contained a large quantity of white powder, which he suspected to be heroin. The counselor telephoned the police, and an officer was dispatched to the school. The counselor handed the officer the envelope, which he transported to the police station. At the station house, the officer opened the envelope and sent a sampling of the substance to the police lab. Tests confirmed the substance to be heroin. The police did not secure a search warrant before confiscating and opening the envelope. The student was thereafter arrested and charged with unlawful possession of a controlled dangerous substance. The student's attorney has filed a motion to suppress the heroin from evidence. The motion will most likely be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}