{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"439fe377-d144-5935-840d-65af39696aeb","task_key":"default--test--5187","task_revision_id":"1","upstream_id":"5187","short_description":"An individual paid taxes 27 months ago, but did not file a tax return for that…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Three years.', 'Four years.', 'Seven years.', 'Within the same fiscal year.', 'One year.', 'Two years.', 'There is no time limit.', 'Within 18 months.', 'Five years.', 'Six months.']\",\"question\":\"An individual paid taxes 27 months ago, but did not file a tax return for that year. Now the individual wants to file a claim for refund of federal income taxes that were paid at that time. The individual must file the claim for refund within which of the following time periods after those taxes were paid?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}