{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ed71421-4124-5a21-b1f6-04df9101f5c8","task_key":"default--test--1802","task_revision_id":"1","upstream_id":"1802","short_description":"A defendant was arrested and charged with involuntary manslaughter for causing a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['denied, because the statement was made during a formal legal proceeding and is therefore admissible.', 'granted, because the questioning of the witness was leading.', 'granted, because the statement is hearsay not within any recognized exception.', \\\"granted, because the witness's statement could prejudice the jury against the defendant.\\\", 'granted, because the statement was not made under oath.', \\\"denied, because the defendant's attorney did not object at the grand jury hearing that the questioning of the witness was leading.\\\", \\\"denied, because the defendant's attorney did not properly cross-examine the witness at the grand jury hearing.\\\", \\\"denied, because the defendant's attorney did not object at the grand jury hearing that the witness's testimony was hearsay.\\\", 'denied, because the defendant was given an opportunity to cross-examine the witness at the grand jury hearing.', \\\"granted, because the witness's statement is irrelevant to the case.\\\"]\",\"question\":\"A defendant was arrested and charged with involuntary manslaughter for causing a vehicular homicide. A grand jury was convened, and various witnesses were subpoenaed. At the grand jury, a witness was asked the following question by the prosecuting attorney: \\\"Is it not true that 20 minutes after the accident you told the police that the defendant's car went through the red light?\\\" He answered, \\\"Yes. \\\"The defendant was also subpoenaed to appear before the grand jury. The defendant was represented by counsel at the grand jury hearing. During questioning he was shown the witness's statement from his testimony. No objection was made to the introduction of the witness's statement. At the defendant's trial, the prosecuting attorney seeks to introduce into evidence the witness's statement from the deposition taken at the grand jury hearing. The defendant's attorney objects to this proffer of evidence. The objection at trial will be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}