{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d807a5f-20c8-5633-a53f-36b52439ce25","task_key":"all--test--8d807a5f-20c8-5633-a53f-36b52439ce25","task_revision_id":"1","upstream_id":"","short_description":"In a prosecution of a defendant for receiving stolen property, an informer…","config":"all","split":"test","body":"{\"choices\":\"['improper, as evidence of conduct not related to truthfulness.', 'improper, as relating to convictions of crimes not punishable by imprisonment in excess of one year.', \\\"proper, as tending to show the informer's bad character for truthfulness.\\\", \\\"proper, as relevant to the informer's possible bias.\\\"]\",\"question\":\"In a prosecution of a defendant for receiving stolen property, an informer testified that the defendant sold him a stolen stereo, which the defendant knew had been stolen. During direct examination, the informer testified that he was unemployed but sometimes worked part-time as a substitute teacher. On cross-examination, the defendant's attorney asked the informer if he had recently filed an application to become a police officer. The informer responded affirmatively. The defendant's attorney then asked the informer the following question:\\\"Isn't it true that you hope that by acting as an undercover agent, the police department will overlook the fact that you had two misdemeanor convictions for possession of marijuana?\\\" The prosecuting attorney immediately objected. The trial judge should rule that the defendant's attorney's inquiry concerning the informer's hopes and misdemeanor convictions is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}