{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b932957-d125-5407-b641-b7d9b1918144","task_key":"default--test--1712","task_revision_id":"1","upstream_id":"1712","short_description":"A driver and passenger were driving to work in the former's automobile one…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"recover nothing, because the passenger's act was a supervening cause.\\\", 'recover only a portion of damages, because both the driver and the owner of the abandoned auto were negligent.', 'recover nothing, because he was distracted while driving.', 'recover all of his damages, because the owner of the abandoned auto was negligent.', 'recover only a portion of damages, because the abandoned auto was in plain view.', 'recover all of his damages, because the abandoned auto was a traffic hazard.', 'recover all of his damages, because the defendant created a dangerous condition.', \\\"recover nothing, because he was driving at a speed that didn't allow him to stop in time.\\\", 'recover nothing, because he had the last clear chance to avoid the collision.']\",\"question\":\"A driver and passenger were driving to work in the former's automobile one morning. As they were traveling at a speed of 20 m. p. h. (which was within the posted speed limit), the passenger suddenly pointed to an overturned vehicle along the side of the highway and said, \\\"Look at that car upside down. \\\" The driver turned to look at the overturned vehicle. As he was looking toward the side of the road, the driver failed to see an abandoned vehicle with a flat tire in the highway about 200 feet in front of his approaching auto. Seconds later, the driver crashed into the rear of the abandoned auto and was injured. The jurisdiction has a relevant comparative negligence statute in effect. If the driver asserts a claim against the owner of the abandoned auto, the most likely result is that the driver will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}