{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0063c5bf-9a12-5388-b2a7-4fb37695c6ce","task_key":"default--test--1696","task_revision_id":"1","upstream_id":"1696","short_description":"A man decided to stop at a drive-through hamburger stand for a late snack. As he…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['successful, because there was a \\\"substantial certainty\\\" that the customer would be hit by the bullet.', 'successful, because the man intended to shoot the manager.', 'successful, because the man caused harm with his actions, regardless of his intent.', \\\"successful, because the bullet from the man's gun directly caused the customer's injury.\\\", \\\"unsuccessful, because the man was provoked by the manager's refusal to serve him.\\\", 'unsuccessful, because the man was not aiming at the customer.', \\\"unsuccessful, because the manager could have prevented the situation by accepting the man's order.\\\", 'unsuccessful, because the man could not foresee that the bullet would hit anyone other than the manager.', 'unsuccessful, because the man did not intend to shoot the customer.', 'successful, because the man was acting recklessly and created a dangerous situation.']\",\"question\":\"A man decided to stop at a drive-through hamburger stand for a late snack. As he drove up to the drive- through line, the manager of the hamburger stand informed him through the intercom system that the restaurant was closing and no further orders would be accepted. She told the man that the last car to be served was the one directly in front of him. The man became angry and yelled into the intercom machine, \\\"Listen, babe, I am hungry. I want two cheeseburgers, a large order of fries, and a Coke. \\\" The manager retorted, \\\"I'm terribly sorry, but we cannot accept your order. \\\"Shortly thereafter, the manager handed the food order to the passengers in the car immediately in front of the man's. When the man saw the manager serving that car, he became very angry, drove his automobile up to the service window and shouted at the manager, \\\"You can't do this to me. \\\" When the manager laughed, the man suddenly reached into the car's glove compartment and pulled out a gun. He aimed at the manager and fired the weapon, intending to hit her. The bullet missed the manager but hit a customer, wounding him in the head. In an action by the customer against the man for battery, the customer will be \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}