{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2483846-b3b3-59ee-a380-f92d2209e87a","task_key":"all--test--e2483846-b3b3-59ee-a380-f92d2209e87a","task_revision_id":"1","upstream_id":"","short_description":"A driver was prosecuted for speeding. It is conceded that the speed limit on the…","config":"all","split":"test","body":"{\"choices\":\"['sustain the objection, because the radar results are not conclusive evidence of speeding.', 'sustain the objection, because radar is not a technique recognized by the scientific community.', 'overrule the objection, because the court will take judicial notice of the accuracy of such tests.', 'overrule the objection, because the radar results would be construed as an admission against interest.']\",\"question\":\"A driver was prosecuted for speeding. It is conceded that the speed limit on the stretch of the highway in question at the time of the alleged speeding was 55 m. p. h. As part of its case-in-chief, the prosecution called a police officer who testified that at about 11:30 a. m. one mile east of the town line, he set up a radar apparatus. The officer testified that, with the use of tuning forks, he found the radar apparatus was accurate. The officer also stated that he had five years of experience in operating such radar devices. Furthermore, the officer testified that the radar's mechanism for recording the speed of vehicles by means of an ink line drawn mechanically on a roll of paper was also functioning properly. Upon further questioning, the officer stated that at about 1:30 p. m. , the needle of the speed indicator dial of the radar apparatus showed the driver's car passing his location at 65 m. p. h. If the driver's attorney makes an objection to the introduction of the radar finding that the driver was traveling at a speed of 65 m. p. h. , the court will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}