{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93ae8b46-421e-5f43-a3a5-f8ecc70f3aed","task_key":"default--test--1349","task_revision_id":"1","upstream_id":"1349","short_description":"Late one night, an accountant walked into a bar and ordered a whiskey sour. The…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"nominal cause of the pedestrian's injuries.\\\", \\\"intervening cause of the pedestrian's injuries.\\\", \\\"contributing cause of the pedestrian's injuries.\\\", \\\"remote cause of the pedestrian's injuries.\\\", \\\"proximate cause of the pedestrian's injuries.\\\", \\\"superseding cause of the pedestrian's injuries.\\\", \\\"incidental cause of the pedestrian's injuries.\\\", \\\"secondary cause of the pedestrian's injuries.\\\", \\\"underlying cause of the pedestrian's injuries.\\\", \\\"direct cause of the pedestrian's injuries.\\\"]\",\"question\":\"Late one night, an accountant walked into a bar and ordered a whiskey sour. The bartender served the drink, even though the accountant looked and acted as though he was already very intoxicated. The accountant soon had consumed five more cocktails, which the bartender served, despite the accountant's obviously and unmistakably drunken condition. After finishing his sixth drink in the bar, the accountant said good night to the bartender, staggered out of the bar, got into his car, and drove away. After weaving back and forth across the road for several blocks, the accountant crashed his car into a pedestrian who was walking on the sidewalk next to the road. The pedestrian suffered serious injuries to his back and legs. The bartender's act of serving drinks to the accountant would most likely be viewed as the\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}