{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0238691-4396-5fc4-b0d3-b87c67f1adcd","task_key":"all--test--e0238691-4396-5fc4-b0d3-b87c67f1adcd","task_revision_id":"1","upstream_id":"","short_description":"A law clerk normally worked from 9:00 a. m. to5:00 p. m. each day, or an average…","config":"all","split":"test","body":"{\"choices\":\"['The pedestrian prevails, because the law firm was negligent for requiring the law clerk to stay at the law office and finish the memorandum.', \\\"The pedestrian prevails, because the law firm is legally responsible for its employees' actions.\\\", \\\"The pedestrian loses, because the actions of the law firm were not a cause-in-fact of the pedestrian's injuries.\\\", \\\"The pedestrian loses, because the law firm is not responsible for the law clerk's negligence in these circumstances.\\\"]\",\"question\":\"A law clerk normally worked from 9:00 a. m. to5:00 p. m. each day, or an average of 40 hours per week. One morning, the law clerk came to the office at 9:00 a. m. and started preparing a brief for an upcoming trial. After completing his work at 5:00 p. m. that afternoon, he was about to leave the office when the senior partner of the law firm summoned him. The senior partner told the law clerk that the law firm was representing an important client on an urgent matter that needed immediate research. The law clerk was advised that he would be required to stay at the law office that night and prepare a memorandum. When the law clerk hemmed and hawed, the senior partner handed him the file and said, \\\"I don't care if you stay all night, but you better have this memo on my desk by 8:00 a. m. tomorrow. \\\"Following the senior partner's instructions, the law clerk stayed at the office until 2:00 a. m. preparing the memorandum. By the time he finished, the law clerk was totally exhausted after having worked a total of 17 hours that day. Afterward, the law clerk left the office and started to drive home. Because of his fatigue, he didn't see a pedestrian crossing the street. His car struck the pedestrian, seriously injuring herSThe pedestrian has asserted a tort action against the law firm to recover damages for her injuries. Which of the following is the most likely result?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}