{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8545ccdd-5e84-55d1-bce7-441c4b1634b5","task_key":"default--test--1332","task_revision_id":"1","upstream_id":"1332","short_description":"A taxicab driver stopped at a convenience store to buy a snack. While he was…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"recover, because the taxi driver's conduct was the proximate cause of the clerk's injuries.\\\", \\\"recover, because the taxi driver's actions were reckless and endangered everyone in the store.\\\", \\\"not recover, because the shooting was not a foreseeable consequence of the taxi driver's conduct.\\\", 'not recover, because the police officer intended to shoot the taxi driver, not the store clerk.', \\\"not recover, because the taxi driver had no control over the police officer's actions.\\\", \\\"recover, because it was foreseeable that the taxi driver's conduct would result in the clerk being shot.\\\", 'recover, because the taxi driver should have known his actions could provoke the police officer.', 'recover, because the taxi driver indirectly caused the police officer to react violently.', \\\"not recover, because the taxi driver didn't physically harm the clerk.\\\", 'not recover, because the police officer was the one who fired the gun.']\",\"question\":\"A taxicab driver stopped at a convenience store to buy a snack. While he was handing his money to the store clerk at the cash register, a police officer entered the store. The taxi driver had received several traffic tickets from the police officer, and he felt that the police officer had been very rude to him. Seeking to gain some revenge, the taxi driver immediately began to taunt and berate the police officer, calling him a variety of profane and insulting names. The police officer was going through a series of personal difficulties and was feeling very emotionally fragile. As the taxi driver's insults rained down on him, the police officer suddenly snapped. He whipped out his gun and fired a shot at the taxi driver. The bullet missed the taxi driver but hit the store clerk, who was standing behind the taxi driver. The clerk survived the shooting but suffered a gunshot wound to his shoulder. In an action by the injured clerk against the taxi driver to recover for his personal injuries, plaintiff will most likely\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}