{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"95409ee6-29fd-5dfe-a8bd-ae739852cef6","task_key":"all--test--95409ee6-29fd-5dfe-a8bd-ae739852cef6","task_revision_id":"1","upstream_id":"","short_description":"A car manufacturer produced a car that was sold nationwide. Problems with the…","config":"all","split":"test","body":"{\"choices\":\"['Federal law, because the judgment was entered in federal court.', 'Federal law, because the judgment was the result of a nationwide action governed by the federal class action rule.', 'State law, because the judgment is being asserted in a state court.', 'State law, because there is no general federal common law and preclusion is a common law doctrine.']\",\"question\":\"A car manufacturer produced a car that was sold nationwide. Problems with the car's brakes allegedly caused several accidents and injuries. Two individual buyers of the car each filed a class action, in different states, against the manufacturer, asserting the same products liability claims on behalf of all buyers nationwide. One class action was filed in federal court and the other was filed in state court. The parties in the federal action reached a court- approved settlement, and the court entered judgment dismissing the action with prejudice. The manufacturer's attorney has moved to dismiss the state court action on the basis of res judicata (claim preclusion). Should the state court look to federal or state law to decide the effect of the judgment?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}