{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2724737c-9c3b-52a1-86f9-98e0364e7b11","task_key":"all--test--2724737c-9c3b-52a1-86f9-98e0364e7b11","task_revision_id":"1","upstream_id":"","short_description":"A federal agent wants to obtain a court order for a wiretap. Which of the…","config":"all","split":"test","body":"{\"choices\":\"['That the wiretap will extend indefinitely until all relevant conversations are recorded.', 'Details on the nature of the offense that has been, is being or will be committed.', 'A particular description of the type of communications sought to be intercepted.', 'Identification of the persons to be overheard must be given, if known.']\",\"question\":\"A federal agent wants to obtain a court order for a wiretap. Which of the following is not required to be set forth in the agent's affidavit submitted to the court for an order authorizing the wiretap?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}