{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4364e1bf-2bcb-58a3-869b-37fa6a0097df","task_key":"default--test--5846","task_revision_id":"1","upstream_id":"5846","short_description":"Markson Co. traded a concrete-mixing truck with a book value of $10,000 to Pro…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Is the loss on the exchange more than the decrease in future cash flows?', 'Is there a significant difference between the book value and fair value of the assets exchanged?', \\\"Does the exchange have a direct impact on the company's net income?\\\", 'Is the fair value of the asset received less than the book value of the asset given up?', 'Does the book value of the asset given up exceed the fair value of the asset received?', 'Are the future cash flows expected to change significantly as a result of the exchange?', 'Is the exchange taxable?', 'Is the gain on the exchange less than the increase in future cash flows?', 'Is the exchange nontaxable?', \\\"Does the exchange result in a change in the company's depreciation schedule?\\\"]\",\"question\":\"Markson Co. traded a concrete-mixing truck with a book value of $10,000 to Pro Co. for a cement-mixing machine with a fair value of $11,000. Markson needs to know the answer to which of the following questions in order to determine whether the exchange has commercial substance?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}