{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4577fc0a-92c5-5bd5-851b-37986ab428a5","task_key":"default--test--1170","task_revision_id":"1","upstream_id":"1170","short_description":"A man was standing on a street corner when a police officer approached him. The…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"granted, because the man didn't have an attorney present during the second statement.\\\", \\\"not granted, because the first statement doesn't affect the legality of the second.\\\", 'not granted, because the second statement was volunteered after a knowing Miranda waiver.', 'not granted, because the man had already confessed to the crime.', 'not granted, because the man willingly gave the second statement.', 'granted, because the first statement was excluded, and the second statement is tainted.', 'granted, because the man was not informed of his rights before the second statement.', 'not granted, because the Miranda warning was correctly given before the second statement.', 'granted, because the man was under duress during the second statement.', 'granted, because the second statement was the fruit of the first statement.']\",\"question\":\"A man was standing on a street corner when a police officer approached him. The officer gently grabbed the man's arm and said, \\\"Hey, I want to talk to you. \\\" The man responded, \\\"About what . . . I'm in a hurry to go home and watch the baseball game. \\\" The officer then remarked, \\\"Listen, what were you doing at the liquor store last week?\\\" The man stated, \\\"All right, man, I was there during the robbery. \\\" Whereupon, the officer placed the man under arrest and charged him with robbery. The man was taken to police headquarters where he was booked. For the first time, he was given his Miranda warnings. He then waived his rights and confessed to being an accessory during the liquor store robbery. Afterward, the court appointed an attorney to represent the man. The attorney moved to suppress both statements the man gave to the police. The trial court granted the motion with respect to the first statement as a violation of Miranda. The man's attorney now moves to suppress the secondstatement given at the police headquarters. This motion should be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}