{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4524b859-86ca-58a4-a30c-739f92005a63","task_key":"default--test--1916","task_revision_id":"1","upstream_id":"1916","short_description":"A defendant was prosecuted for mail fraud. At trial, the defendant moved to have…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the agentÕs testimony might be influenced by the testimony of other witnesses.', 'No, provided that the government can show that the agentÕs presence is essential to the presentation of its case.', 'Yes, because the defendant has a right to exclude all persons who may be called as government witnesses.', \\\"Yes, as the agent's potential bias could affect the fairness of the trial.\\\", \\\"No, as the agent's role in the investigation makes his presence necessary for legal proceedings.\\\", 'No, because the government has a right to have its designated representative remain in the courtroom through-out the trial.', 'Yes, to prevent the possibility of the agent influencing other witnesses.', 'Yes, as all witnesses, regardless of their role, should be excluded to ensure impartiality.', \\\"No, as the agent's presence does not necessarily influence the outcome of the trial.\\\"]\",\"question\":\"A defendant was prosecuted for mail fraud. At trial, the defendant moved to have all witnesses excluded from the courtroom, and the court granted the motion. The government named the investigating FBI agent as its designated representative. Upon learning that the agent would be giving testimony during the trial, the defendant moved that the agent also be excluded from the courtroom. Should the defendantÕs motion be granted?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}