{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"103a1fa1-f7a2-5c2c-bfe3-f3df68cf1a36","task_key":"default--test--1010","task_revision_id":"1","upstream_id":"1010","short_description":"One afternoon, police officers observed a man and a14-year-old girl smoking…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the man, the adult principal, must be convicted before any prosecution can be maintained against a minor.', 'she was forced by the man to smoke marijuana.', 'she was unaware of the consequences of her actions due to her age.', 'the girl did not know the substance was marijuana, thus she cannot be held accountable.', 'since the statute was designed to protect minors, the girl cannot be prosecuted as an accomplice.', 'the girl was under the influence and thus not fully conscious of her actions.', 'smoking marijuana does not necessarily make her an accomplice to the crime.', 'a minor cannot be prosecuted for an adult crime.', 'the girl cannot be prosecuted as an accomplice because she was also a victim.', 'the girl did not supply the marijuana, thus she cannot be held responsible.']\",\"question\":\"One afternoon, police officers observed a man and a14-year-old girl smoking marijuana together. They arrested the man and charged him with the separate offenses of (1) possession of a controlled dangerous substance, and (2) contributing to the delinquency of a minor. The girl was also arrested and charged with being an accomplice to the crime of contributing to the delinquency of a minor. At trial, the girl's best defense is that\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}