{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"73161048-5d42-5c82-9841-657ea208fbf7","task_key":"default--test--7114","task_revision_id":"1","upstream_id":"7114","short_description":"Which of the following is most likely to decrease the demand for kindergarten…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['An increase in online learning programs for young children', 'Increased demand for pre-school teachers', 'More high school graduates choosing to go straight to work rather than furthering their education.', 'Increased investments in high school education', 'An increase in funding for education', 'A decrease in the population of school-aged children', 'A decrease in the average number of children per household', 'Increased immigration of foreign citizens and their families', 'Subsidies given to college students who major in elementary education', 'More parents choosing to homeschool their children']\",\"question\":\"Which of the following is most likely to decrease the demand for kindergarten teachers?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}