{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1a23d18-063d-5730-a466-d979d79c4bf8","task_key":"all--test--e1a23d18-063d-5730-a466-d979d79c4bf8","task_revision_id":"1","upstream_id":"","short_description":"While performing certain nonaudit services for an insurance company, a…","config":"all","split":"test","body":"{\"choices\":\"['Agree to recommend the appropriate accounting treatment after performing sufficient research on weather hedges.', 'Refuse to conduct the research and make a recommendation, because of insufficient experience.', 'Refuse to conduct the research and make a recommendation, because of a conflict of interest.', 'Agree with the accounting treatment recommended by the company’s hedge fund trader.']\",\"question\":\"While performing certain nonaudit services for an insurance company, a professional accountant is asked to recommend the appropriate accounting treatment for a weather hedge transaction. The accountant has worked with financial hedges but has no experience with weather hedges. Which of the following actions by the accountant would be in compliance with the IFAC Code of Ethics for Professional Accountants?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}