{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84fbfa57-3f89-5b03-9108-ffe6c313db55","task_key":"default--test--949","task_revision_id":"1","upstream_id":"949","short_description":"A state has a hit-and-run statute in effect that makes it a crime to leave the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['guilty, because she failed to report the accident to the authorities.', 'guilty, because she unlawfully fled the scene of an accident in violation of her statutory duty.', 'guilty, because she should have anticipated the potential for further accidents.', \\\"guilty, because her actions directly resulted in the victim's death.\\\", \\\"not guilty, because there was not a sufficient causal connection between her actions and the victim's death to impose criminal responsibility.\\\", \\\"guilty, because her failure to render aid to the victim would make the defendant criminally responsible for the victim's death.\\\"]\",\"question\":\"A state has a hit-and-run statute in effect that makes it a crime to leave the scene of an accident. One afternoon, a 9-year-old victim was riding her bicycle along the street. As the victim swerved into the southbound lane, her bicycle was struck by a car driven by the defendant. The victim was knocked off her bike and thrown onto the sidewalk adjacent to the street. Although the victim received some minor scrapes and bruises, she was not seriously injured. Following the accident, the car driven by the defendant sped away. Moments later, however, a tractor-trailer crashed into the rear of a car about 30 feet from where the victim was lying. The car almost instantly caught fire as its gas tank exploded. The victim, who was engulfed in the flaming wreckage, was killed. If the defendant is charged with involuntary manslaughter for the death of the victim, the defendant should be found\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}