{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24e083f7-e280-5645-bd62-b3a1bdaca2ff","task_key":"all--test--24e083f7-e280-5645-bd62-b3a1bdaca2ff","task_revision_id":"1","upstream_id":"","short_description":"Markson Co. traded a concrete-mixing truck with a book value of $10,000 to Pro…","config":"all","split":"test","body":"{\"choices\":\"['Does the book value of the asset given up exceed the fair value of the asset received?', 'Is the gain on the exchange less than the increase in future cash flows?', 'Are the future cash flows expected to change significantly as a result of the exchange?', 'Is the exchange nontaxable?']\",\"question\":\"Markson Co. traded a concrete-mixing truck with a book value of $10,000 to Pro Co. for a cement-mixing machine with a fair value of $11,000. Markson needs to know the answer to which of the following questions in order to determine whether the exchange has commercial substance?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}