{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4db98225-9f61-5df8-8173-544c5220a126","task_key":"default--test--1919","task_revision_id":"1","upstream_id":"1919","short_description":"Defendant is on trial. Which of the following, if occurs, would NOT cause a due…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The defendant is not provided with a competent interpreter when needed.', 'The defendant is not given the opportunity to testify on their own behalf.', 'Jurors are exposed to influence favorable to the prosecution.', 'The trial is held in a location that is prejudiced against the defendant.', 'The defendant is compelled to stand trial or appear at penalty phase proceedings visibly shackled (unless there are security concerns).', 'The defendant is denied the opportunity to present evidence.', \\\"The defendant's past convictions are disclosed to the jury before they reach a verdict.\\\", 'The defendant is not allowed to cross-examine witnesses.', 'The defendant is compelled to stand trial in street clothing.', 'The trial is conducted in a way that jurors would not be able to give evidence reasonable consideration.']\",\"question\":\"Defendant is on trial. Which of the following, if occurs, would NOT cause a due process violation in the trial?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}