{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41a60884-d9e4-58db-9dac-d08dc3fe1e40","task_key":"default--test--11133","task_revision_id":"1","upstream_id":"11133","short_description":"Which of the following is evidence that de Marneffe cites in support of the…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['Marijuana use increased in states where it was legalized.', 'Methamphetamine use increased during periods when it was readily available.', 'The percentage of physicians who use psychoactive drugs is much higher than the general population.', 'Heroin use was much higher among military personnel in Vietnam than it was among veterans who returned to the United States.', 'The number of smokers increased after the introduction of e-cigarettes.', 'all of the above', 'Cocaine use decreased in areas where it was strictly prohibited.', 'Alcohol abuse declines when the law restricts the times it can be legally sold.', 'The use of prescription drugs is higher in countries where they are easily accessible.', 'The consumption of caffeine increased with the rise of coffee shops.']\",\"question\":\" Which of the following is evidence that de Marneffe cites in support of the claim that increased availability of a drug leads to increased use of that drug?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}