{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6d72fc7-25af-542e-865c-c482719808df","task_key":"default--test--5910","task_revision_id":"1","upstream_id":"5910","short_description":"What is the due date of a federal estate tax return (Form 706), for a taxpayer…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['October 15, year 2.', 'April 15, year 3.', 'January 31, year 3.', 'November 15, year 2.', 'December 31, year 2.', 'May 15, year 3.', 'March 15, year 3.', 'February 15, year 3.', 'September 15, year 2.', 'August 15, year 2.']\",\"question\":\"What is the due date of a federal estate tax return (Form 706), for a taxpayer who died on May 15, year 2, assuming that a request for an extension of time is not filed?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}