{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c89444e-a798-5b01-9cc3-739d1f51de17","task_key":"default--test--1730","task_revision_id":"1","upstream_id":"1730","short_description":"A shopper suffered injuries to her back and legs when she slipped on a loose…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"admissible, because it shows the owner's responsibility for maintaining safe conditions.\\\", \\\"inadmissible, because the owner's actions after the accident do not pertain to the conditions at the time of the accident.\\\", 'inadmissible, because it is evidence of a good deed by the owner after the accident.', \\\"inadmissible, because it is speculation about the owner's control over the hallway.\\\", 'admissible, because it proves the owner had the power to make changes in the hallway.', 'admissible, because it shows that the owner retained ownership and control.', \\\"admissible, because it demonstrates the owner's negligence in not maintaining the hallway previously.\\\", \\\"admissible, because it is relevant to the issue of the owner's knowledge of the unsafe condition.\\\", 'inadmissible, because the evidence is irrelevant to the accident.']\",\"question\":\"A shopper suffered injuries to her back and legs when she slipped on a loose piece of carpeting in a hallway connecting a department store and a cafeteria. The shopper brought suit against the owner of the entire building. The owner claimed that since he leased the cafeteria, he no longer retained any control over the connecting hallway. If the shopper offers into evidence the fact that two days after the accident, the owner had the hallway carpet re-laid, the court should rule that the evidence is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}