{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0ba51ec-91d0-57f8-a12d-8b95e939dfff","task_key":"default--test--1335","task_revision_id":"1","upstream_id":"1335","short_description":"A defendant was arrested for the armed robbery of a liquor store. The defendant…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The defendant had a private lawyer on retainer at the time he gave his statement.', 'The defendant was not informed about the charges against him before the confession.', 'The defendant was denied food and water before the confession.', 'Before the defendant gave the statement he was refused permission to use the bathroom.', \\\"The defendant's arrest was not based on probable cause.\\\", 'The defendant was physically abused by the police before the confession.', 'The defendant was under the influence of alcohol during the confession.', 'The defendant was not allowed to have his lawyer present during the interrogation.', 'The defendant could not speak English, and the warnings were given in English.', 'The defendant was not read his Miranda rights until after he confessed.']\",\"question\":\"A defendant was arrested for the armed robbery of a liquor store. The defendant was taken directly from the scene of the crime in a patrol car to the police station where he was placed, still handcuffed, in an interview room. He waited for about an hour, and a detective finally came in. The detective gave the defendant his Miranda warnings, and the defendant agreed to sign the waiver and speak to the detective. The defendant then confessed to the robbery. Which of the following allegations would be LEAST helpful in suppressing his statement?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}