{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5ff0c86-0110-5e0f-99bf-12aedf696a54","task_key":"default--test--1388","task_revision_id":"1","upstream_id":"1388","short_description":"A mail clerk domiciled in State A slipped and fell on ice that had formed near…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the employer cannot intervene in a personal injury case.', \\\"No, because the employer's claim is not directly related to the clerk's personal injury claim.\\\", 'Yes, because the employer also has a direct negligence claim against the building owner.', \\\"No, because although the employer has an interest in the clerk's action, that interest is not likely to be impaired in the employer's absence.\\\", \\\"Yes, because the employer has a financial interest in the outcome of the clerk's case.\\\", \\\"No, because the employer's claim exceeds the amount the clerk is seeking in damages.\\\", 'Yes, because the employer is an indispensable party.', \\\"Yes, because the employer's claim and the clerk's claim arose from the same incident.\\\", \\\"Yes, because the employer's claim shares common questions of law and fact with the clerk's action.\\\", 'No, because the clerk chose not to join the employer as a co-plaintiff in his action.']\",\"question\":\"A mail clerk domiciled in State A slipped and fell on ice that had formed near the loading dock of the building in State B where the clerk's State B employer leased space for its headquarters. The building was owned and operated by a State C corporation. As a result of the fall, the clerk was injured and the employer's expensive computer he was carrying was badly damaged. The clerk sued the building owner for negligence in a federal district court in State B, seeking $100,000 in personal-injury damages. The employer has filed a timely motion to intervene, asserting an $80,000 negligence claim against the building owner for the damage to its computer. Is the court likely to grant the employer's motion to intervene?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}