{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0ee7d73-2c49-52e1-bf76-b7cb1e50185e","task_key":"all--test--c0ee7d73-2c49-52e1-bf76-b7cb1e50185e","task_revision_id":"1","upstream_id":"","short_description":"The attorney for a plaintiff in an action filed in federal district court served…","config":"all","split":"test","body":"{\"choices\":\"['Interrogatories are only proper to discover facts, not contentions.', 'Interrogatories may not be served until an answer to the complaint is filed.', 'Interrogatories may not be served until the parties have conferred to arrange for initial disclosures and prepare a discovery plan.', 'The interrogatories exceed the number permitted without permission from the court or an agreement between the parties.']\",\"question\":\"The attorney for a plaintiff in an action filed in federal district court served the defendant with the summons, the complaint, and 25 interrogatories asking questions about the defendant's contentions in the case. The interrogatories stated that they were to be answered within 30 days after service. The defendant is likely to succeed in obtaining a protective order on which of the following grounds?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}