{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f0111ee-6d65-5498-82fa-017d8f779d09","task_key":"default--test--1613","task_revision_id":"1","upstream_id":"1613","short_description":"A man and a woman owned adjoining pieces of land. The man moved a giant magnolia…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['denied, because the woman has a right to protect her property from potential harm.', \\\"granted, because the man had not intentionally caused harm to the woman's property.\\\", 'denied, because hurricanes were common in the area.', 'granted, because the man cannot control where a tree falls during a hurricane.', 'denied, because the man should have taken precautions to secure the tree during a hurricane.', \\\"granted, because the woman's damages resulted from an act of God.\\\", 'granted, because the woman knew the risk of hurricanes in the area when she built her greenhouse.', 'denied, because the trier of fact may still infer liability for trespass.', 'granted, because the woman did not produce any evidence to show that the man was negligent.', \\\"denied, because the man knew the risk of planting a tree so close to the woman's property.\\\"]\",\"question\":\"A man and a woman owned adjoining pieces of land. The man moved a giant magnolia tree from another section of his property and had it replanted on his property at a spot just 10 feet from the woman's property. At that time, the woman protested that by locating the tree so close to her land, the man was increasing the risk of injury to the woman's greenhouse, in which the woman cultivated prizewinning flowers. Hurricanes are quite common in the area. Severalyears after the replanting of the tree, in the midst ofa hurricane, the tree fell on the woman's greenhouse,destroying rare and valuable plants valued at$25,000. The woman asserted a claim against the man to recover damages for the harm caused by the tree falling on the greenhouse. During trial, the only evidence the woman presented was that the hurricane uprooted the tree, causing it to fall onto her property, and thereby resulting in the damage as claimed. At the end of the woman's case, the man moved for a directed verdict. The man's motion will most likely be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}