{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"192d1e54-2726-51b3-98a4-eefb153328ff","task_key":"all--test--192d1e54-2726-51b3-98a4-eefb153328ff","task_revision_id":"1","upstream_id":"","short_description":"A defendant was convicted of fraud after a jury trial in state court. The…","config":"all","split":"test","body":"{\"choices\":\"['Nothing more, because the unjustifiable failure to object to important prosecution evidence is structural error.', 'That the attorney was court-appointed and not privately retained.', \\\"That there is a reasonable probability that the trial's outcome would have been different if the attorney had objected.\\\", \\\"That there is clear and convincing evidence that the trial's outcome would have been different if the attorney had objected.\\\"]\",\"question\":\"A defendant was convicted of fraud after a jury trial in state court. The conviction was affirmed on direct appeal. The defendant timely moved for postconviction relief under the Sixth Amendment on the ground that his attorney had provided ineffective assistance of counsel. The trial judge, after a hearing, found that the attorney had performed deficiently by failing to raise a proper objection that would have resulted in exclusion of important prosecution evidence. What more, if anything, must the trial court find in order to sustain the defendant's Sixth Amendment claim?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}