{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"55a75884-11aa-59ac-a900-6511d16df684","task_key":"all--test--55a75884-11aa-59ac-a900-6511d16df684","task_revision_id":"1","upstream_id":"","short_description":"A 16-year-old boy was bullied regularly at school by several older boys. The…","config":"all","split":"test","body":"{\"choices\":\"['Yes, children aged 14 and older are treated as adults and, as adults, they are subject to the same capital punishment penalty as other adults.', 'No, a minor under 16 cannot be tried in adult court because the Eighth Amendment clause against cruel and unusual punishment forbids it, and the whole prosecution was unconstitutional.', \\\"Yes, when a crime is committed by malice aforethought, the age of the perpetrator doesn't matter because the same criminal intent requires the same punishment.\\\", 'No, the Court applied the bright-line test that forbids capital punishment of youth under the age of 18 because there are distinct differences between an adolescent and an adult.']\",\"question\":\"A 16-year-old boy was bullied regularly at school by several older boys. The 16-year-old was developmentally disabled and his mannerisms were a magnet for malicious bullies. One day the boy stole his father's licensed handgun and shot one of the bullies dead. The district attorney announced prosecution of the teen as an adult and asked for the death penalty due to aggravating circumstances. The trial court certified the boy over to adult criminal court. The boy was convicted and sentenced to death. An appeal was taken on his behalf all the way to the U.S. Supreme Court. The Supreme Court agreed to hear the case on the merits. Will the Court affirm the sentence of death?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}