{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ee822d1-cfba-5e6c-a3e7-851f569fc9d5","task_key":"all--test--9ee822d1-cfba-5e6c-a3e7-851f569fc9d5","task_revision_id":"1","upstream_id":"","short_description":"A state recidivist statute made it a felony punishable by life imprisonment…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the gravity of the sentence is disproportionate to the seriousness of the crime, none of which involved bodily injury, and there is no possibility of parole.', 'Yes, because life imprisonment can never be an appropriate punishment for criminality that does not take human life.', 'No, because theft and passing bad checks has become a national emergency warranting tough remedial action to protect the economy, institutions and persons being victimized.', \\\"No, because it is improper for the courts to attempt to interject their value judgments into a state's ostensibly reasonable legislative sentencing schemes.\\\"]\",\"question\":\"A state recidivist statute made it a felony punishable by life imprisonment without parole to be convicted for the fifth time of any form of criminal theft involving goods or money of $100 or more. A man's record showed four prior convictions for nonviolent crimes involving theft or bad checks, each one pertaining to property or cash in excess of $100 but not in excess of $500. On the fifth conviction, after conviction of passing a $100 bad check, the court sentenced him to life without parole under a repeat offender statute. He challenged the sentence and the statute based on the Eighth Amendment prohibition against cruel and unusual punishment. The U.S. Supreme Court granted certiorari. Will the Court likely invalidate the sentence on these facts?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}