{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6fdf2b0-99ab-5a65-8849-f32f9fff61a2","task_key":"default--test--1066","task_revision_id":"1","upstream_id":"1066","short_description":"An investor sued a corporation for stock fraud. In presenting his case-in-chief,…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admitted, because it is relevant to the case.', 'admitted, under the business records exception.', 'excluded, because it is considered hearsay.', 'excluded, because the newspaper is not a credible source.', 'admitted, under the past recollection recorded exception.', 'excluded, because it violates the original document rule.', 'excluded, because the newspaper copy does not fit within any established exception to the hearsay rule.', 'excluded, because it violates the best evidence rule.', 'admitted, under the market reports exception.']\",\"question\":\"An investor sued a corporation for stock fraud. In presenting his case-in-chief, the investor sought to introduce an issue of a newspaper to show the corporation's stock price on that given day. Upon objection by the corporation's attorney, this evidence should be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}