{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fdc39b94-989f-5114-bc16-35367c61658e","task_key":"default--test--1944","task_revision_id":"1","upstream_id":"1944","short_description":"During spring break, a private boarding school was deserted while students and…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['granted, because the student was not present during the search.', 'denied, because the school has a policy allowing searches of student rooms.', 'granted, because the police did not have probable cause to test the substance.', 'granted, because the police should have secured a warrant before opening the envelope.', 'denied, because the search was conducted by a private party.', \\\"granted, because the student's room is considered private property.\\\", 'granted, because the police should have secured a warrant before seizing the envelope.', 'denied, because the counselor had a reasonable suspicion of illegal activity.', \\\"denied, because the discovery of the substance was incidental to the counselor's actions.\\\", 'denied, because the counselor, as a school employee, was in loco parentis.']\",\"question\":\"During spring break, a private boarding school was deserted while students and teachers were away on vacation. A guidance counselor remained on campus because he was working on a research project. After working late one night, the counselor decided to enter the room of a student from a very wealthy family. The counselor was rummaging through the student's room looking for something valuable to steal. Under the bed, he noticed an expensive suitcase. The counselor opened the suitcase and found an express mail envelope. The counselor knew that the student's father often sent money to his son in express mail envelopes. The counselor opened the envelope and saw that it contained a large quantity of white powder, which he suspected to be heroin. The counselor telephoned the police, and an officer was dispatched to the school. The counselor handed the officer the envelope, which he transported to the police station. At the station house, the officer opened the envelope and sent a sampling of the substance to the police lab. Tests confirmed the substance to be heroin. The police did not secure a search warrant before confiscating and opening the envelope. The student was thereafter arrested and charged with unlawful possession of a controlled dangerous substance. The student's attorney has filed a motion to suppress the heroin from evidence. The motion will most likely be \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}