{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2113e8e-800c-5374-913c-b4900b0c2802","task_key":"default--test--2150","task_revision_id":"1","upstream_id":"2150","short_description":"A ten-year-old boy, tested with a paper-and-pencil intelligence test, obtained…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['130, but with a significant decrease in subtest scores', 'more than 130', '140, indicating significant intellectual growth', '130', '130, but with a different pare of subtest scores', 'less than 130', '130, but with a significant increase in subtest scores', '100, showing a significant drop in intelligence quotient', '120, indicating a decrease in intellectual capabilities', 'Unpredictable as the test used was different']\",\"question\":\"A ten-year-old boy, tested with a paper-and-pencil intelligence test, obtained an IQ score of 130. Two years later, he was tested with an alternate form of the same test. Most likely, the obtained score on the second testing would be\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}