{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b9e5757-7342-5da0-9df3-b8e7f66b968f","task_key":"default--test--1070","task_revision_id":"1","upstream_id":"1070","short_description":"A defendant was charged with armed robbery. The defendant had only an…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the defendant was not adequately informed of his right to counsel.', \\\"No, because the trial court's error was harmless in light of the overwhelming evidence.\\\", \\\"No, because the defendant made a conscious decision to represent himself after the judge's refusal.\\\", 'No, because the defendant explicitly expressed his desire to represent himself, indicating a valid waiver.', 'Yes, because the record does not establish a valid waiver of the right to counsel.', 'No, because the defendant had the opportunity to request another lawyer but chose not to.', \\\"Yes, because the defendant's lack of legal knowledge and low level of education made self-representation unfair.\\\", 'Yes, because the defendant was not capable of effectively representing himself.', 'No, because the defendant voluntarily waived his right to counsel.', 'Yes, because the defendant was not given the opportunity to have another lawyer appointed.']\",\"question\":\"A defendant was charged with armed robbery. The defendant had only an eighth-grade education and no legal knowledge. At a pretrial hearing, the defendant told the judge that he was unhappy with the public defender who had been appointed to represent him and that he wanted the judge to appoint another lawyer. The judge refused to appoint another lawyer, telling the defendant, with no further explanation, that his only other choice was to represent himself. The defendant said that he would represent himself at trial then, as \\\"anything [was] better than having this lawyer.\\\" The defendant did not raise the subject again, and he represented himself at trial. After hearing overwhelming evidence of the defendant's guilt, the jury convicted him of armed robbery. On appeal, the defendant has requested a new trial on the ground that he was deprived of his right to counsel. Should the appellate court grant the defendant's request?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}