{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66398201-4436-52e4-932e-288dc20b95fa","task_key":"default--test--1719","task_revision_id":"1","upstream_id":"1719","short_description":"A plantiff sued a defendant for injuries suffered m a fall on the sidewalk of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['inadmissible, because it is hearsay evidence.', \\\"admissible, because the physician had personal knowledge of the plaintiff's injuries.\\\", \\\"admissible, because it provides additional information about the plaintiff's condition.\\\", \\\"admissible, because it sheds light on the plaintiff's physical condition at the time of the fall.\\\", \\\"inadmissible, because the physician's testimony is not relevant to prove that the plaintiff's alleged injuries are false or exaggerated.\\\", \\\"inadmissible, because the physician's testimony is biased.\\\", \\\"inadmissible, because the defendant's counsel failed to lay a proper foundation.\\\", \\\"inadmissible, because it contradicts the defendant's statement.\\\", 'admissible, because it is reasonably pertinent to diagnosis or treatment.', 'admissible, because it helps to establish the timeline of events.']\",\"question\":\"A plantiff sued a defendant for injuries suffered m a fall on the sidewalk of the defendant's home. The plaintiff's complaint alleged that the walk was covered by a thick sheet of ice, which had been negligently left there for several days by the defendant. In his answer, the defendant set forth that the ice formed overnight and that the plaintiff fell before the defendant had a chance to remove it. During the trial, a physician, whose office was next door to the defendant's home, testified that he saw the plaintiff fall and that the ice, which had been there for several days was at least two inches thick. On cross-examination, counsel for the defendant asked the physician the following question: \\\"During your treatment of the plaintiff on the day in question, is it not true that he told you his fall resulted from a loss of equilibrium after he suffered from dizziness?\\\"Upon objection by the plaintiff's attorney, the physician's testimony will be ruled\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}