{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96e1c258-1097-5caa-a0d6-2bd754ac5d72","task_key":"all--test--96e1c258-1097-5caa-a0d6-2bd754ac5d72","task_revision_id":"1","upstream_id":"","short_description":"A mail clerk domiciled in State A slipped and fell on ice that had formed near…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because although the employer has an interest in the clerk's action, that interest is not likely to be impaired in the employer's absence.\\\", 'No, because the clerk chose not to join the employer as a co-plaintiff in his action.', 'Yes, because the employer is an indispensable party.', \\\"Yes, because the employer's claim shares common questions of law and fact with the clerk's action.\\\"]\",\"question\":\"A mail clerk domiciled in State A slipped and fell on ice that had formed near the loading dock of the building in State B where the clerk's State B employer leased space for its headquarters. The building was owned and operated by a State C corporation. As a result of the fall, the clerk was injured and the employer's expensive computer he was carrying was badly damaged. The clerk sued the building owner for negligence in a federal district court in State B, seeking $100,000 in personal-injury damages. The employer has filed a timely motion to intervene, asserting an $80,000 negligence claim against the building owner for the damage to its computer. Is the court likely to grant the employer's motion to intervene?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}