{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"313182ff-eca6-5280-b797-be14cb9a26c7","task_key":"default--test--1071","task_revision_id":"1","upstream_id":"1071","short_description":"A young man suggested to his friend that they steal a large-screen TV from a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No crime.', 'Conspiracy and larceny.', 'Aiding and abetting a burglary.', 'Burglary.', 'Conspiracy.', 'Accessory after the fact.', 'Accessory before the fact.', 'Attempted larceny.', 'Assault.', 'False imprisonment.']\",\"question\":\"A young man suggested to his friend that they steal a large-screen TV from a neighborÕs house. The friend was angry with the young man and decided to use the opportunity to get even with him by having him arrested. The friend said he would help, and that night, he drove the young man to the neighborÕs house. The young man broke in while the friend remained out-side. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The jurisdiction defines crimes as at common law. Of what crime, if any, can the friend properly be convicted?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}