{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eaa01507-93fa-5c3f-ad0c-2c3fd816a375","task_key":"default--test--1022","task_revision_id":"1","upstream_id":"1022","short_description":"A defendant is on trial for attempted murder. The alleged victim is called by…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible as a statement of then existing condition.', 'inadmissible, because it is a statement of opinion rather than fact.', 'inadmissible, because the victim did not make the statement under oath.', 'inadmissible, because the victim was not questioned about the statement before being excused.', 'admissible as a present sense impression.', 'inadmissible because the witness was not present during the attack.', 'admissible as an excited utterance.', 'inadmissible as hearsay not within any recognized exception.', 'admissible as a statement against interest.']\",\"question\":\"A defendant is on trial for attempted murder. The alleged victim is called by the prosecution to testify. During her testimony, the victim recounted the incident and described how the defendant savagely beat her with a baseball bat. She was not asked by the prosecution whether she made any statements during the attack. After the victim was excused and left the witness stand, the prosecution called another witness to testify. The witness proposes to testify that when the beating stopped, the victim screamed, \\\"I'm dying. Don't let the defendant get away with this. \\\"Upon objection by the defendant's attorney, the witness's proffered testimony is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}