{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9fa56d2d-a93d-53e7-8f9f-f364f16c3504","task_key":"default--test--1414","task_revision_id":"1","upstream_id":"1414","short_description":"A farmer owns a large farm on which he allows his friends to hunt during…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, because the farmer's friends were the ones responsible for the shooting.\\\", 'Yes, because the farmer had a duty to ensure the safety of his property.', \\\"No, because the farmer is not responsible for his friends' conduct.\\\", 'No, because the neighbor was trespassing.', 'Yes, because the careless friend was permitted to hunt without safety training.', 'No, because the farmer was not present at the time of the accident.', 'Yes, because the use of firearms is an abnormally dangerous activity.', 'Yes, because the farmer should have posted warning signs about hunting activities.', 'No, because the neighbor should have known of the danger due to the hunting season.', 'Yes, because the farmer allowed unsafe hunting practices on his property.']\",\"question\":\"A farmer owns a large farm on which he allows his friends to hunt during quail-hunting season. He does not provide his friends with any instructions about gun safety. The neighbor who owns property adjacent to the farm knows of the friends' use of the property during the hunting season. One day during the hunting season, without the farmer's knowledge or permission, the neighbor took a shortcut across the farm to visit an acquaintance. The neighbor was wounded by a shot fired by one of the farmer's friends, who was shooting at quail and carelessly failed to see the neighbor. Traditional rules of landowners' and occupiers' liability apply. In an action by the neighbor against the farmer to recover for the injuries, will the neighbor be likely to prevail? \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}