{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0e1b2ac7-cba3-50cd-a424-a3cee221b251","task_key":"default--test--5984","task_revision_id":"1","upstream_id":"5984","short_description":"Which of the following explanations most likely would satisfy an auditor who…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Plant assets were retired during the current year.', \\\"Prior years' depreciation expenses were erroneously overstated.\\\", 'Prior years’ depreciation expenses were erroneously understated.', 'The company revalued its plant assets during the current year.', 'The estimated depreciation method was changed during the current year.', 'Current year’s depreciation expense was erroneously understated.', 'The salvage value of the plant assets was revised upward.', 'The estimated remaining useful lives of plant assets were revised upward.', 'The company made significant purchases of plant assets in the current year.']\",\"question\":\"Which of the following explanations most likely would satisfy an auditor who questions management about significant debits to accumulated depreciation accounts in the current year?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}