{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02c4d4e0-f5ca-5f2e-85a7-5024084135ca","task_key":"default--test--915","task_revision_id":"1","upstream_id":"915","short_description":"A resident in an exclusive residential area is a marine biologist. To aid in his…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['not recover, because she suffered injury only because she had a cold and, therefore, was unusually vulnerable to harm from being splashed with water.', 'not recover, because the resident had taken all necessary precautions to ensure the shark could not harm anyone.', 'recover, because the resident intentionally placed the shark in an area where it could potentially harm others.', \\\"not recover, because the development of bronchitis was not a direct result of the resident's actions.\\\", 'recover, because the possessor of wild animals is strictly liable for all injuries caused to others by the animals.', \\\"not recover, because she willingly chose to visit the resident's property and view the shark tank.\\\", 'not recover, because she did not suffer the type of harm normally inflicted by a shark.', 'recover, because the smell from the shark tank was a nuisance.', 'recover, because the resident knowingly kept a dangerous wild animal in a residential area.', 'recover, because the resident failed to warn her of the potential risk of being splashed by the shark.']\",\"question\":\"A resident in an exclusive residential area is a marine biologist. To aid in his study of sharks, he had a large tank built in his backyard in which he placed a great white shark. Aside from a smell that emanated from the tank, some neighbors were concerned and afraid to let their children outside for fear that they would wander onto the resident's property. In order to convince his neighbors that they had nothing to fear, the resident invited them over to view the shark tank. While a neighbor was standing near the tank, the shark splashed its tail, and the neighbor got very wet. The neighbor, who had a cold, developed bronchitis. If the neighbor sues the resident for damages incurred from being drenched by the shark and bases her suit on strict liability, she will most likely\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}