{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39e363a1-9215-53cb-b538-786fc79f7f16","task_key":"default--test--1876","task_revision_id":"1","upstream_id":"1876","short_description":"A tenant lived in a small apartment building that was owned by a landlord. In…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible as a written admission by the party opponent.', \\\"admissible, because it corroborates the tenant's claims.\\\", \\\"inadmissible, because the accountant's testimony is not the best evidence.\\\", \\\"inadmissible under the Dead Man's Statute.\\\", 'admissible as secondary evidence under the circumstances.', 'inadmissible, because the accountant is an interested party.', 'admissible, because it falls under the business record exception.', 'inadmissible, because it constitutes hearsay evidence.', 'inadmissible, due to the lack of physical contract as evidence.', 'admissible as an exception to the hearsay rule.']\",\"question\":\"A tenant lived in a small apartment building that was owned by a landlord. In July, the landlord was killed in an accident. In August, the tenant brought an action against the administrator of the landlord's state. The tenant alleged that he and the landlord had a contract under the terms of which the landlord was supposed to pay him $50 a week to mow the lawn and perform small maintenance repairs for the other tenants. He claimed that the landlord had not paid him for the past six months. He testified that there was a written contract and that the landlord kept the only copy of it in his home. At trial, he called the landlord's accountant to testify about the contract. The accountant testified that she had seen the contract. She further testified about the terms of the contract, consistent with the tenant's testimony. She testified that the contract, along with many other documents pertaining to the landlord's business, had most likely been destroyed when the landlord's business office was robbed a few months earlier. The trial judge should rule that the accountant's. testimony is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}