{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0bcae1f5-e865-5cbd-9fe4-c79e79ddc4c1","task_key":"all--test--0bcae1f5-e865-5cbd-9fe4-c79e79ddc4c1","task_revision_id":"1","upstream_id":"","short_description":"Late one night, an accountant walked into a bar and ordered a whiskey sour. The…","config":"all","split":"test","body":"{\"choices\":\"[\\\"proximate cause of the pedestrian's injuries.\\\", \\\"superseding cause of the pedestrian's injuries.\\\", \\\"direct cause of the pedestrian's injuries.\\\", \\\"intervening cause of the pedestrian's injuries.\\\"]\",\"question\":\"Late one night, an accountant walked into a bar and ordered a whiskey sour. The bartender served the drink, even though the accountant looked and acted as though he was already very intoxicated. The accountant soon had consumed five more cocktails, which the bartender served, despite the accountant's obviously and unmistakably drunken condition. After finishing his sixth drink in the bar, the accountant said good night to the bartender, staggered out of the bar, got into his car, and drove away. After weaving back and forth across the road for several blocks, the accountant crashed his car into a pedestrian who was walking on the sidewalk next to the road. The pedestrian suffered serious injuries to his back and legs. The bartender's act of serving drinks to the accountant would most likely be viewed as the\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}