{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b73f1e4d-cc0f-5ac6-81f4-cfbe14d9bc15","task_key":"default--test--1899","task_revision_id":"1","upstream_id":"1899","short_description":"A teenager is the star player on his high school basketball team. The high…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the school, because drug testing is not an intrusion on his Fourth Amendment right of privacy.', 'the school, because the student is a public figure and therefore has less expectation of privacy.', 'the student, because the coach admitted that he did not have any individualized suspicion that the student was under the influence of drugs.', \\\"the school, because the student's drug use is a risk to his health and the safety of others.\\\", 'the school, because a urine sample does not constitute a search under the Fourth Amendment.', \\\"the student, because the school's policy of random drug testing is not enforceable.\\\", \\\"the student, because the school's policy of random drug testing is a violation of his civil rights.\\\", 'the student, because the school did not have a legitimate reason to suspect him of drug use.', \\\"the student, because he didn't consent to the drug test.\\\"]\",\"question\":\"A teenager is the star player on his high school basketball team. The high school is a public school, and the school has a policy that all student athletes are required to sign an authorization form by which they consent to undergo random drug testing. All the basketball players on the school team, except the teenager, signed the authorization. In the third game of the season, the team beat their archrival, and the teenager scored a record high 69 points. In the locker room following the game, the basketball coach approached the teenager and requested that he provide a urine sample for a random drug test. The teenager refused. He had smoked some marijuana the previous day and was afraid that a positive test result might jeopardize his being awarded a basketball scholarship to college. Thereafter, the teenager was suspended from the team for failing to comply with the coach's request. The coach later admitted that he did not have any individualized suspicion that the teenager was under the influence of drugs. The teenager has filed a motion challenging the suspension and seeking reinstatement on the basketball team. The court will most likely rule in favor of\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}