{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51373cd9-99af-51c7-868f-c2833e50964b","task_key":"all--test--51373cd9-99af-51c7-868f-c2833e50964b","task_revision_id":"1","upstream_id":"","short_description":"A shop owner domiciled in State A sued a distributor in a federal district court…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the wholesaler's and the distributor's principal places of business are diverse.\\\", \\\"No, because there is supplemental jurisdiction over the wholesaler's counterclaim.\\\", 'Yes, because there is no diversity of citizenship between the distributor and the wholesaler.', 'Yes, because there is no diversity of citizenship between the shop owner and the wholesaler.']\",\"question\":\"A shop owner domiciled in State A sued a distributor in a federal district court in State A for breach of a contract. The shop owner sought $100,000 in damages for allegedly defective goods that the distributor had provided under the contract. The distributor is incorporated in State B, with its principal place of business in State C. The distributor brought in as a third-party defendant the wholesaler that had provided the goods to the distributor, alleging that the wholesaler had a duty to indemnify the distributor for any damages recovered by the shop owner. The wholesaler is incorporated in State B, with its principal place of business in State A. The wholesaler has asserted a $60,000 counterclaim against the distributor for payment for the goods at issue, and the distributor has moved to dismiss the counterclaim for lack of subject-matter jurisdiction. Should the motion to dismiss be granted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}