{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1739d59d-8ddb-503b-bc67-b8bcb1164acb","task_key":"all--test--1739d59d-8ddb-503b-bc67-b8bcb1164acb","task_revision_id":"1","upstream_id":"","short_description":"A first-year law student worked as a part-time law clerk at a prestigious law…","config":"all","split":"test","body":"{\"choices\":\"['The law firm had failed to give him his salary for two weeks, and he took the book as security for the debt.', 'He intended to return the book after reading it.', 'The law firm had given him possession of the book.', 'The law firm had given him custody of the book.']\",\"question\":\"A first-year law student worked as a part-time law clerk at a prestigious law firm. He was permitted to borrow law books and legal treatises to take home for his own personal use. One afternoon, the law student was preparing a brief in the firm's law library when he noticed a book lying on the conference table. Making sure no one was watching him, he took the book and placed it in his briefcase. He planned to take the book home that night with the intention of not returning it. If the law student is subsequently charged with larceny of the book, which of the following claims, if true, would LEAST aid him in his defense?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}