{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3cc117cd-25e1-541c-9187-19e515e9e5b5","task_key":"all--test--3cc117cd-25e1-541c-9187-19e515e9e5b5","task_revision_id":"1","upstream_id":"","short_description":"A patient domiciled in State A sued a surgeon domiciled in State B in a federal…","config":"all","split":"test","body":"{\"choices\":\"['No, because the appellate court lacks jurisdiction over the appeal.', \\\"No, because the district court's decision on jurisdiction is final.\\\", 'Yes, because a contrary appellate decision could terminate the action.', \\\"Yes, because the surgeon's personal- jurisdiction challenge raises a constitutional question.\\\"]\",\"question\":\"A patient domiciled in State A sued a surgeon domiciled in State B in a federal court in State A, alleging claims for malpractice. The surgeon moved to dismiss the action for lack of personal jurisdiction. The court denied the motion and set discovery cutoff and trial dates. The surgeon has appealed the denial of the motion. Should the appellate court hear the merits of the surgeon's appeal?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}