{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00ca0b2c-8cd6-5f2c-b1ee-27f9faef6d3c","task_key":"default--test--1911","task_revision_id":"1","upstream_id":"1911","short_description":"A defendant was arrested and charged with possession of heroin. At the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['denied, because there is no basis for a double jeopardy challenge.', 'granted, because the heroin possession charge was formally dismissed at the preliminary hearing.', 'denied, because double jeopardy rights do not attach unless there has been an acquittal or conviction.', 'granted, because the defendant has the right to avoid self-incrimination.', 'denied, because the grand jury proceeding is not considered a trial.', 'granted, because by voluntarily appearing at the grand jury hearing, the defendant has standing to challenge the indictment.', 'granted, because the defendant has already been tried for the same charge.', 'denied, because double jeopardy only applies if the initial charge resulted in a conviction or acquittal.', 'granted, because the defendant was previously held in custody for the same charge.']\",\"question\":\"A defendant was arrested and charged with possession of heroin. At the defendant's preliminary hearing, the prosecution presented evidence that the defendant was arrested while riding in a car. The heroin was found in a briefcase containing no identification, but several papers found inside contained references to the defendant. The defendant's attorney argued that the briefcase did not belong to the defendant, and the defendant had no idea how the heroin got inside the briefcase. The court ruled that there was insufficient probable cause to support a prosecution of the defendant on the heroin possession charge. The defendant was released from custody and charges were dropped. Several weeks later, a grand jury was convened to determine if the defendant should be re-indicted for heroin possession based on the same alleged incident reviewed at the preliminary hearing. The defendant was called to testif' as a witness at the grand jury hearing. He appeared and was sworn in as a witness. The defendant proceeded to challenge the proceeding on the basis that any grand jury indictment for heroin possession would constitute a violation of the double jeopardy clause. The defendant's double jeopardy challenge should be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}