{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9168555f-c5de-502f-a933-2d5a44afac3c","task_key":"default--test--1336","task_revision_id":"1","upstream_id":"1336","short_description":"One afternoon after school, a 14-year-old girl went into a drug store. While…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"the girl's statement was made in a public place.\\\", 'the girl was not in custody.', 'the girl was not under arrest at the time.', \\\"the security guard's remark was an accusation, not an interrogation.\\\", 'the girl voluntarily admitted to the act.', 'the security guard did not force the girl to speak.', \\\"the girl's statement was not coerced or influenced.\\\", 'Miranda warnings need not be given to juveniles.', \\\"the security guard's testimony provides crucial evidence.\\\", 'the security guard was not employed by an agency of the government.']\",\"question\":\"One afternoon after school, a 14-year-old girl went into a drug store. While browsing, she picked up a candy bar and placed it in her coat pocket. She left the store without paying for it. Unknown to the girl, the store security guard saw what she had done. He immediately ran outside, grabbed her arm, and accused her of stealing the candy bar. The girl broke down and started crying. She then told the security guard that she was hungry because she hadn't eaten any lunch. At a juvenile delinquency court hearing, the security guard proposes to testify to what the girl said. Upon objection by the girl's attorney, the strongest reason for admitting her statement will be because\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}