{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27628361-f589-5e79-9429-f6ff71d9b1c0","task_key":"default--test--1980","task_revision_id":"1","upstream_id":"1980","short_description":"A state statute made it a crime to be addicted to the use of narcotic drugs. A…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the statute makes it criminal to use illegal drugs, which constitutes cruel and unusual punishment.', 'No, because the arrest is based on the regular use of illegal narcotics and is a proper concern of the criminal law.', 'Yes, because the physical evidence is not enough to prove that the defendant is currently using drugs.', 'No, because the jury found him guilty based on the evidence presented in the trial.', 'Yes, because addiction is a medical condition and should be treated as such, not criminalized.', \\\"No, because the state has the power to forbid the use of narcotics within its borders and that's what this statute does.\\\", 'Yes, because the statute punishes having the illness of drug addiction, which is like punishing someone for having a cold, and it is thus cruel and unusual punishment.', 'No, because the defendant admitted to past drug usage, hence he is guilty.', 'No, because the state has the right to protect its citizens from the harmful effects of narcotics.']\",\"question\":\"A state statute made it a crime to be addicted to the use of narcotic drugs. A man was arrested and charged with the crime of drug addiction. Two officers testified that they examined \\\"track\\\" marks on the defendant's arms and wrists, and that he admitted to past drug usage. The accused testified that he was not an addict and the marks were from an allergic reaction. The trial court instructed the jury that it could find him guilty on proof that he was an addict, without proof of drug usage. The jury did not believe the defendant's denials, and found him guilty based on the observed track marks. The court sentenced him to six months imprisonment. He appealed, claiming cruel and unusual punishment prohibited by the Eighth and Fourteenth Amendments of the United States Constitution. Will the conviction likely be reversed on appeal?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}