{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35493464-558a-5671-9ee0-99ead8d0320f","task_key":"default--test--1180","task_revision_id":"1","upstream_id":"1180","short_description":"After weeks of deliberation, a boyfriend decided to embark on his plan to rob a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Murder.', 'No crime.', 'Involuntary manslaughter.', 'Negligent homicide.', 'Conspiracy to commit murder.', 'Attempted murder.', 'Second degree murder.', 'Accessory after the fact.', 'Voluntary manslaughter.']\",\"question\":\"After weeks of deliberation, a boyfriend decided to embark on his plan to rob a bank. As part of his scheme, the boyfriend enlisted his girlfriend, who agreed to drive the getaway car. On the day of the robbery, the girlfriend and the boyfriend drove to the bank. After the girlfriend parked outside and the boyfriend went into the bank, she saw a security guard in the bank and changed her mind and fled on foot, leaving the keys in the car with the motor running. A few minutes later, the boyfriend went into the bank, robbed it, and then ran back to the car. He drove off and was speeding away from the bank when he looked behind to see if he was being followed. As he took his eye off the road, the boyfriend's vehicle struck a pedestrian who was crossing the street, killing her. For the pedestrian's death, the girlfriend should be found guilty of which, if any, of the following crimes?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}