{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1960ff6a-872f-5ddd-af45-fe7581e853ef","task_key":"default--test--1454","task_revision_id":"1","upstream_id":"1454","short_description":"A man was paroled after serving five years in prison for forgery. Three weeks…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['He had planned to dispose of the gun in a safe manner.', 'He was driving to the police station to give the gun to the authorities when the officer stopped his car.', 'He had previously served as a police officer.', 'He found the gun just moments before the officer stopped his car.', 'He did not know about the federal statute.', 'He had no previous history of violent crime.', 'He had a license to carry a firearm from another state.', 'He was unaware that the object he found was a functioning firearm.', 'His previous conviction did not involve the use of a gun or other weapon.', 'He did not intend to use the gun for an unlawful purpose.']\",\"question\":\"A man was paroled after serving five years in prison for forgery. Three weeks later, he found a handgun in a high school parking lot. Fearing that students from the school might find the gun and get into trouble using it, the man put it in the trunk of his car. The man drove off, was lawfully stopped by a police officer for speeding, and allowed the officer to search his car and the trunk. During the search, the officer discovered the gun. The man was charged under a federal statute prohibiting the knowing possession of a firearm by a convicted felon. Which of the following additional facts, if established, would be most helpful to the man's defense?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}