{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22f2f3da-da1f-52a2-8b6f-b94a469adca8","task_key":"default--test--1345","task_revision_id":"1","upstream_id":"1345","short_description":"A student at a private university was receiving financial aid from the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The university did not have a written policy on the termination of financial aid due to honor code violations.', 'The student was not provided with a copy of the honor code at the time of his enrollment.', 'The honor committee did not have any student representatives.', \\\"The university did not re-allocate the student's financial aid to another deserving student.\\\", 'The honor committee sent the student a letter to advise him of the hearing date.', 'The professor who caught the student cheating did not provide any evidence to the honor committee.', 'The university did not provide any guidelines on what constitutes an honor code violation.', 'The university was in financial difficulties and could not meet its payroll expenses.', 'The university received support from the state.']\",\"question\":\"A student at a private university was receiving financial aid from the university based on a financial aid application he completed at the time of his enrollment. During finals at the end of the fall semester, the student was caught cheating. His chemistry professor saw him looking at a fellow student's exam paper and copying the answers to three multiple choice questions. The professor notified the honor committee, which immediately found an honor code violation and ordered that the student receive a failing grade in the course. In addition, in accordance with the rules of the university, the student's financial aid was terminated. The student filed a lawsuit seeking reinstatement of his financial aid. The student's only argument was that his due process rights were violated in that he did not receive a notice of, or a hearing on, the alleged honor code violation. Which of the following facts, if true, would be most helpful to the student?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}