{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a2ad08c-086f-5709-87a8-8b6bb9d5f908","task_key":"default--test--1819","task_revision_id":"1","upstream_id":"1819","short_description":"In a suit by an investor against a speculator, a relevant fact is the price of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible as a public record.', 'admissible, to prove the authenticity of the newspaper.', 'admissible under the library records exception to the hearsay rule.', 'admissible as a business record.', 'admissible as past recollection recorded.', 'inadmissible as hearsay evidence.', 'admissible, to prove the price of gold on that day.', 'inadmissible as not the best evidence.', 'inadmissible due to lack of original copy.']\",\"question\":\"In a suit by an investor against a speculator, a relevant fact is the price of gold as quoted on a specific date 20 years ago. The investor calls a librarian to authenticate the microfilm copy of a newspaper from that date. This copy, kept in the archives of the public library, is the only record of that particular issue of the newspaper. The microfilm is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}