{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dd50600-07dc-52b4-99a2-fb773c4934c6","task_key":"all--test--4dd50600-07dc-52b4-99a2-fb773c4934c6","task_revision_id":"1","upstream_id":"","short_description":"Long was a purchasing agent for Frost, a sole proprietor. Long had express…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Yes, because Long's agency survived Frost's incompetence.\\\", \\\"Yes, because the order was valid regardless of Frost's incompetence.\\\", 'No, because Grant dealt only with Long.', 'No, because Long did not have authority to enter into the contract.']\",\"question\":\"Long was a purchasing agent for Frost, a sole proprietor. Long had express authority to place purchase orders with Frost's suppliers. Long conducted business on the Internet and had little contact with Frost. After Frost was declared incompetent in a judicial proceeding, Long placed an order with Grant Corp. on Frost's behalf. Both Long and Grant were unaware of Frost's incompetence. Will Frost or Frost's legal representative be liable to Grant?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}