{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb273323-3108-5af0-a785-787232e2e37d","task_key":"default--test--1285","task_revision_id":"1","upstream_id":"1285","short_description":"During a deer-hunting season open to rifle hunters, a hunter saw a deer in the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the hunter was negligent in not ensuring the area was clear before shooting.', 'No, because the hunter did not intend to shoot the hiker.', 'No, because the hiker should have made his presence known to the hunter.', 'Yes, because the hunter intentionally shot the rifle.', 'No, because the hunter did not make direct physical contact with the hiker.', 'No, because the hunter was aiming at a deer, not a human.', 'Yes, because the hunter had a duty to ensure safety while hunting.', \\\"Yes, because the hiker suffered injuries due to the hunter's action.\\\", 'No, because the hiker was in the forest during hunting season.']\",\"question\":\"During a deer-hunting season open to rifle hunters, a hunter saw a deer in the forest. He shot his rifle at the deer, hoping to hit and kill it. Instead, he hit and injured a hiker. The hunter had not realized that the hiker was there. Does the injured hiker have an actionable battery claim against the hunter?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}