{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9162d421-b35d-56bb-b47d-c0acb1ea23e0","task_key":"all--test--9162d421-b35d-56bb-b47d-c0acb1ea23e0","task_revision_id":"1","upstream_id":"","short_description":"A waitress sued her former employer, a restaurant, under Title VII in federal…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because a challenge to jurisdiction can be made at any time, even after judgment has been entered, and here the jurisdictional requirement was missing.', \\\"Yes, because an appellate court has no authority to reverse a lower court's finding that there was no subject-matter jurisdiction.\\\", 'No, because a judgment entered on a jury verdict can never be changed or attacked after a judgment is entered.', 'No, because this is a challenge to the existence of a cause of action, i.e., the failure to state a claim, which does not go to jurisdiction and cannot be raised post-judgment.']\",\"question\":\"A waitress sued her former employer, a restaurant, under Title VII in federal court. She claimed that the owner had sexually harassed her and then caused her wrongful termination. A jury returned a verdict of $100,000 for the former employee. Two weeks after the court entered judgment on the verdict, the employer filed a motion to dismiss under Fed. Rule 12(b)(6), claiming that it did not have at least 15 employees as required by the definition of an \\\"employer\\\" under the civil rights statute. The employer claimed that there was therefore no federal subject-matter jurisdiction. The lower court agreed, dismissed the case and voided the judgment. On appeal to the U.S. Circuit Court of Appeals, will the appellate court likely uphold the lower court's decision?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}