{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"239c3dd1-e544-5e3c-841f-ae79aebac43b","task_key":"default--test--5405","task_revision_id":"1","upstream_id":"5405","short_description":"A company recently moved to a new building. The old building is being actively…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"It will be removed from the company's assets.\\\", 'It will be classified as a current asset.', 'It will be valued at the estimated selling price.', 'It will be reclassified as an asset held for sale.', 'It will be valued at historical cost.', 'It will no longer be depreciated.', 'It will be classified as an intangible asset.', \\\"It will still be used for the company's operations.\\\", 'It will be considered an investment property.']\",\"question\":\"A company recently moved to a new building. The old building is being actively marketed for sale, and the company expects to complete the sale in four months. Each of the following statements is correct regarding the old building, except:\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}