{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5346725e-ea7a-5c37-9539-7da1469ac4e4","task_key":"default--test--1803","task_revision_id":"1","upstream_id":"1803","short_description":"A person owned property next to a highway. After raking leaves on his property…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"the driver will prevail, if he can prove that the landowner's actions directly resulted in the accident.\\\", 'the driver will prevail, because the landowner should have anticipated the wind change.', 'the landowner will prevail, if the driver cannot prove that the smoke caused the accident.', 'the driver will prevail, if the smoke from the burning leaves prevented him from clearly seeing the roadway.', 'the driver will prevail, because the landowner will be strictly liable for causing the accident.', 'the landowner will prevail, because burning leaves on private property is not against the law.', 'the landowner will prevail, because he called the weather bureau before starting the fire.', 'the driver will prevail, because the landowner should have placed warnings on the highway.', 'the landowner will prevail, because the driver was driving in excess of the speed limit.', 'the landowner will prevail, if his decision to burn the leaves was reasonable under the circumstances.']\",\"question\":\"A person owned property next to a highway. After raking leaves on his property into a large pile, the landowner loaded the leaves into several large metal barrels so that he could burn the leaves. Before starting the fire, the landowner telephoned the local weather bureau to determine which direction the wind would be blowing. Because a highway was located on the southern edge of his property, he was aware that it would be unsafe to burn the leaves if the wind was blowing in that direction. The weather bureau, however, indicated that the wind would be gusting in a northerly direction. Shortly after the landowner set the leaves on fire, the wind current unexpectedly shifted and started gusting in a southerly direction. As a consequence, the smoke and ashes blew over the highway, resulting in poor visibility. Moments later, a motorist was driving his automobile on the highway in a westerly direction. The posted speed limit was 45 m. p. h. , although the driver was traveling about 55 m. p. h. The driver could not see the highway clearly and crashed his vehicle into a median strip. If the driver asserts a claim against the landowner, the most likely result is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}