{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90bfe267-e73b-58fd-9286-05d906efad5f","task_key":"default--test--1040","task_revision_id":"1","upstream_id":"1040","short_description":"A woman was in a public restroom at a mall. While the woman was at the sink, a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Intimidation.', 'Assault.', 'Stalking.', 'Assault with intent to commit rape.', 'Harassment.', 'False imprisonment.', 'Battery.', 'Kidnapping.', 'No crime.', 'Trespassing.']\",\"question\":\"A woman was in a public restroom at a mall. While the woman was at the sink, a man entered the restroom and told her not to make a sound. Frightened, the woman ran to an open window in the bathroom and jumped onto the fire escape. As she exited down the fire escape, the woman accidentally fell, bruising her legs. Which of the following crimes, if any, would the man most likely be convicted of?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}