{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"375b6a36-d294-5df5-8cac-04d52e389fb5","task_key":"default--test--1716","task_revision_id":"1","upstream_id":"1716","short_description":"A teenage girl was babysitting a two-year-old baby. She took the baby in a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, she will be charged with involuntary manslaughter, as her negligence indirectly led to the death of the baby.', 'No, the babysitter did not commit a criminal act under the circumstances \\\\x80\\\\x94 her negligence was not gross enough to rise to the level of criminality and she had no criminal intent.', \\\"Yes, she will be charged with manslaughter due to negligence, as she fell asleep while responsible for the baby's safety.\\\", 'Yes, she would be charged with negligent homicide because it was foreseeable that if she fell asleep for even just a few seconds, the baby would likely be kidnapped and killed.', 'No, because she did not do anything wrong in dozing off and she had no duty to protect the baby at the expense of her own life.', \\\"Yes, this is felony murder in that the babysitter's unacceptable behavior was committed during the commission of the kidnapping felony.\\\", 'No, as the babysitter was threatened and intimidated, she is considered a victim in this situation and will not be charged.', 'Yes, she will be charged with child endangerment as she was negligent in her duties as a babysitter.']\",\"question\":\"A teenage girl was babysitting a two-year-old baby. She took the baby in a stroller to the public park, as she often did. On this day, she negligently \\\"nodded off\\\" for just a few seconds on a park bench with the stroller next to her. She felt some movement and snapped awake to see a man taking the baby from the stroller. The man was large and menacing-looking. The babysitter, who is thin and petite, froze and did not put up a struggle as the man stated, \\\"I will kill you if you move.\\\" The babysitter did scream and call for help as the kidnapper fled with the child. Within a few yards he tripped and dropped the baby on its head. The baby died at the hospital of massive head injuries. Is it probable that the prosecutor will charge the babysitter with a crime, and if so, what will be the charge?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}