{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa8dcdaa-4597-5bc5-bd69-316d2528b045","task_key":"default--test--1018","task_revision_id":"1","upstream_id":"1018","short_description":"A very drunk man was sitting in a bar drinking. A businessman customer came and…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Involuntary Manslaughter', 'Second Degree Murder', 'Voluntary Manslaughter', 'Aggravated Assault', 'Justifiable Homicide', 'First Degree Murder', 'Negligent Homicide', 'Self-Defense', 'Attempted Murder', 'Felony Murder']\",\"question\":\"A very drunk man was sitting in a bar drinking. A businessman customer came and sat beside him. They did not know each other. The businessman had slightly grazed against the drunk man's body. \\\"Man, are you trying to kill me or something?'' said the very drunk man. The businessman did not think anything of it, since he had barely grazed the drunkard. The drunk continued to bully the businessman, calling him made-up names, and saying things that were nonsensical to the businessman and to others at the bar who were listening to the drunk's boisterous and irrational rants. The drunk then took a gun out of his pocket and shot the businessman, killing him. At trial, the defense argued that the man could not form the specific intent to premeditate for first degree murder. Which of the following crimes is the defendant most likely guilty of under these facts?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}