{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10851b0a-7d9d-5f52-8652-94937051e11b","task_key":"default--test--1648","task_revision_id":"1","upstream_id":"1648","short_description":"A 50-year-old nurse who had been fired from his job at ahospital told his…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because sanctions are not proper against the attorney of a represented party.', 'Yes, because the attorney should have known that age discrimination could not be proven in this case.', 'No, because the hospital failed to give the attorney the chance to withdraw the complaint in advance of filing the motion with the court.', \\\"No, because the nurse's complaint was reasonable and thus cannot be sanctioned.\\\", 'Yes, because the attorney did not consider the age of the replacement nurse before filing the complaint.', 'No, because sanctions are only proper when a complaint is filed with malicious intent.', 'No, because the attorney was acting in the best interests of his client.', 'Yes, because the nurseÕs complaint contained legal contentions that were not warranted by existing lawbased on the facts in this case.', 'Yes, because the nurseÕs attorney failed to conduct areasonable pre-filing inquiry.', 'Yes, because the hospital has the right to fire an employee without giving a reason.']\",\"question\":\"A 50-year-old nurse who had been fired from his job at ahospital told his attorney, ÒI was fired because of my age, and I want to sue the hospital.Ó Based on this information, the attorney filed an age discrimination complaint against the hospital in federal court. As it turned out, the hospital had hired a 52-year-old man as the nurseÕs replacement, a fact that rendered an age discrimination claim unavailable. The hospital responded tothe complaint by filing a motion for sanctions against thenurseÕs attorney. Is the court likely to grant the hospitalÕs motion?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}