{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ded82670-f0d7-53c4-8e71-9a146d254708","task_key":"default--test--1192","task_revision_id":"1","upstream_id":"1192","short_description":"A high school student was running for president of the student council. He was a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, the school exercised its appropriate function of trying to restrict the use of vulgarity and offensive discourse in a high school setting where free speech may be limited.', \\\"Yes, in today's society sexual images and sexual humor is so intertwined into the fabric of society that it is something that has to be tolerated in the interest of free speech.\\\", 'Yes, the Supreme Court upheld the relief granted to the student because it was a personal expression and did not incite violence or hate.', \\\"Yes, the student's rights to free speech should be upheld regardless of the content of his speech.\\\", 'Yes, despite some inconvenience to teachers and students, the immature presentation must be tolerated in deference to the need to provide platforms for free speech to students.', \\\"Yes, the school should not have the power to restrict any form of speech as it impedes on the student's rights to express themselves.\\\", 'No, the student had no rights to free speech, especially considering the violent disruptions in the schools in recent years.']\",\"question\":\"A high school student was running for president of the student council. He was a top scholar who had been selected to be the class valedictorian. Inexplicably, he gave a campaign speech at a school assembly that was filled with graphic stories and anecdotes about the imagined sexual adventures and prowess of certain teachers and students. Several teachers and students complained. The principal suspended him for five days and retracted the valedictorian honors. The student, with his parents, sued in federal court, seeking an injunction and damages based on denial of First Amendment free speech rights. The federal courts ruled for the student but the case came before the U.S. Supreme Court for final decision. Did the Supreme Court uphold the relief granted to the student?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}