{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dbd0898b-35b2-52b0-ba9e-e22f334fd70d","task_key":"all--test--dbd0898b-35b2-52b0-ba9e-e22f334fd70d","task_revision_id":"1","upstream_id":"","short_description":"An auditor is selecting prenumbered purchase orders for testing an entity's…","config":"all","split":"test","body":"{\"choices\":\"['Represents office equipment ordered and never received.', 'Has been properly voided in the normal course of business.', 'Represents office equipment ordered and canceled before being processed by the vendor.', 'Has been electronically deleted from the purchase order file.']\",\"question\":\"An auditor is selecting prenumbered purchase orders for testing an entity's internal control activities related to their proper approval before office equipment is ordered. The auditor is matching random numbers with the purchase order numbers to determine which purchase orders to inspect. If a random number matches a voided purchase order the auditor ordinarily would replace the voided purchase order with another if the voided purchase order\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}