{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0ec4dadd-2df4-5c6a-bd76-834c1bee2cfc","task_key":"default--test--1172","task_revision_id":"1","upstream_id":"1172","short_description":"A student at a private university sued the university in fed-eral court for…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Abuse of discretion.', 'Probable cause.', 'Harmless error.', 'Preponderance of the evidence.', 'Inevitable discovery.', 'Manifest injustice.', 'Fair and reasonable.', 'De novo.', 'Substantial evidence.', 'Clearly erroneous.']\",\"question\":\"A student at a private university sued the university in fed-eral court for negligence after he fell from scaffolding in a university-owned theater building. At trial, after briefing from both parties, the court permitted the jury to hear testimony that there had been several previous accidents in the same building. The jury found for the student, and the university appealed. One of the university's arguments on appeal is that the testimony about the previous accidents should have been excluded as irrelevant and highly prejudicial. Which standard of review applies to this argument?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}