{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f8abafbe-94de-5f68-89a6-a9f62f3893ad","task_key":"all--test--f8abafbe-94de-5f68-89a6-a9f62f3893ad","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff filed a tort action in state court but then failed to prosecute the…","config":"all","split":"test","body":"{\"choices\":\"['No, because the judgment was entered by a state court, not a federal court.', 'No, because the state court did not rule on the merits in its dismissal.', 'Yes, because a dismissal with prejudice operates as a judgment on the merits.', 'Yes, because a judgment for failure to prosecute operates as a judgment on the merits under the Federal Rules of Civil Procedure.']\",\"question\":\"A plaintiff filed a tort action in state court but then failed to prosecute the action. The defendant moved to dismiss the action, and the court granted the motion in an order that stated: \\\"The defendant's motion to dismiss is granted, and this action is dismissed with prejudice.\\\" The court accordingly entered judgment for the defendant. The plaintiff then filed the same claim against the defendant in federal court, invoking diversity jurisdiction. The defendant has asserted the defense of res judicata (claim preclusion) in its answer. Should the federal court give preclusive effect to the state court judgment?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}