{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eac78e33-2f72-51d4-b9dd-6e2fb2846295","task_key":"default--test--1890","task_revision_id":"1","upstream_id":"1890","short_description":"A rancher owned a 2,000-acre cattle ranch in a remote section of the state. For…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"the legal cause of the cowboy's death, but not the cause-in-fact.\\\", \\\"both the cause-in-fact and a reckless act resulting in the cowboy's death.\\\", \\\"a negligent act resulting in the cowboy's death, but not the cause-in-fact.\\\", \\\"a reckless disregard for the safety of others, resulting in the cowboy's death.\\\", \\\"both an unforeseeable event and the legal cause of the cowboy's death.\\\", \\\"the cause-in-fact of the cowboy's death, but not the legal cause.\\\", \\\"an intentional act causing the cowboy's death, but not the cause-in-fact.\\\", \\\"an unforeseeable event leading to the cowboy's death, but not the legal cause.\\\", \\\"the legal cause and the cause-in-fact of the cowboy's death.\\\", \\\"neither the legal cause nor the cause-in-fact of the cowboy's death.\\\"]\",\"question\":\"A rancher owned a 2,000-acre cattle ranch in a remote section of the state. For some time the rancher's cattle had been seized upon by roaming packs of wolves. In order to protect his livestock from these attacks, the rancher erected a fence around his property. In addition, the rancher installed electrical wiring on the fence. The wiring transmitted an electric current, which would emit a shock upon contact with the fence. Although the voltage was calculated to frighten animals away from the fence, it was of such low intensity that it posed no risk of injury to a human being, even a small child. Consequently, the rancher did not post any notice of the electrical wiring device. One day, a cowboy was riding his horse along the outskirts of the rancher's property, when a gust of wind blew his hat onto the fence. The cowboy, who had undergone major heart surgery, wore a pacemaker. When the cowboy touched the top of the fence to retrieve his hat, the electric current short- circuited his pacemaker, resulting in the cowboy's death by electrocution. In a wrongful death action by the cowboy's estate against the rancher, the installation of the electrical wiring device would most likely constitute\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}