{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1164bd3-fc41-5d6e-b10e-adf5ff966ab6","task_key":"default--test--1969","task_revision_id":"1","upstream_id":"1969","short_description":"A witness lived next door to a victim. Late one night, the witness overheard the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible as a past recollection recorded if the witness can confirm the statement was made.', 'admissible as a dying declaration if the jury determines that the victim believed that her death was imminent.', 'admissible as a dying declaration if the judge, by preponderance of the evidence, determines that the victim believed that her death was imminent.', 'inadmissible, because the probative value is substantially outweighed by the danger of unfair prejudice.', 'inadmissible, because the statement was not made under oath.', 'inadmissible as hearsay not within any recognized exception.', 'inadmissible, because the statement is considered a character evidence.', 'admissible, because the statement is a present sense impression.', 'inadmissible, because the statement can lead to a character attack on the defendant.']\",\"question\":\"A witness lived next door to a victim. Late one night, the witness overheard the victim scream, \\\"The defendant, please don't do it!\\\" The next day the victim was found dead in her bedroom. The defendant was arrested and charged with murdering the victim. At trial, the witness proposes to testify to the victim's statement. Upon objection by defendant's counsel, the court should rule the witness's testimony regarding the victim's statement\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}