{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75953121-505f-5d6c-8b5f-9c040b13e088","task_key":"all--test--75953121-505f-5d6c-8b5f-9c040b13e088","task_revision_id":"1","upstream_id":"","short_description":"A defendant, a 14-year-old male, was tried as an adult for an armed robbery and…","config":"all","split":"test","body":"{\"choices\":\"['No, because a juvenile may be sentenced to life without parole if release would pose a danger to the community.', 'No, because the Eighth Amendment does not apply outside the context of capital punishment.', 'Yes, because a life-without-parole sentence is not permissible for a juvenile defendant convicted of a nonhomicide offense.', 'Yes, because a life-without-parole sentence is not permissible for an adult or juvenile defendant convicted of a nonviolent offense.']\",\"question\":\"A defendant, a 14-year-old male, was tried as an adult for an armed robbery and was convicted. No one had been physically harmed during the robbery. In light of the defendant's lengthy juvenile criminal history, which included violent offenses, and the fact that armed robbery is a crime of violence, he was sentenced to life in prison without the possibility of parole. Did the life-without-parole sentence violate the Eighth Amendment ban on cruel and unusual punishment?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}