{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a94124c-e50a-5629-b877-9bf68481c111","task_key":"default--test--1363","task_revision_id":"1","upstream_id":"1363","short_description":"A defendant is on trial for theft of a used car that he took for a test drive…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Yes, as it provides context to the defendant's actions.\\\", 'Yes, as a prior consistent statement of the defendant.', 'No, because it is a statement that could potentially influence the jury.', \\\"Yes, as a statement of the defendant's future intentions.\\\", 'No, because it is irrelevant to the case.', 'Yes, as a statement by the defendant of his then-existing state of mind.', 'No, because it is a self-serving statement by an accused.', \\\"Yes, as it is evidence of the defendant's character.\\\", 'No, because it is a statement made after the event.', 'No, because it is hearsay not within any exception.']\",\"question\":\"A defendant is on trial for theft of a used car that he took for a test drive and did not return. He was arrested in the car two days later. In his defense, the defendant has testified that he had no intention of keeping the car but got caught up in marital problems and simply delayed returning it. The defendant calls a witness to testify that the defendant told him, during the two days, \\\"I'm going to return this car as soon as I work things out with my wife.\\\" Is the witness's testimony admissible?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}