{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4530feb4-fc6f-54ad-8581-92481b303267","task_key":"default--test--1315","task_revision_id":"1","upstream_id":"1315","short_description":"A buyer filed a lawsuit against a seller based on a written contract allegedly…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['inadmissible, because the seller contests the authenticity of the written contract.', 'inadmissible, because the best evidence rule requires that the original contract always be used as evidence.', 'admissible, because the duplicate is a carbon copy.', 'admissible, because the original contract was lost.']\",\"question\":\"A buyer filed a lawsuit against a seller based on a written contract allegedly executed at the time of the sale of the seller's hot dog stand to the buyer. Under the terms of the contract, the seller agreed not to open a hot dog stand for a period of two years within a distance of two miles of the old hot dog stand, now owned by the buyer. The buyer alleged in his complaint that the seller violated the contract by opening a hot dog stand one-half mile from the old location. The original contract was lost prior to trial. At trial, the buyer wants to introduce a duplicate carbon copy of the written contract. The seller objects as to the authenticity of the original written contract. The copy is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}