{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56a64097-78d8-5771-81f1-1731d6711003","task_key":"default--test--5820","task_revision_id":"1","upstream_id":"5820","short_description":"A kindergarten teacher has students determine the following. ‚Ä¢ How many…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Measuring', 'Predicting', 'Patterning', 'Comparing', 'Counting', 'Reasoning', 'Sequencing', 'Classifying', 'Ordering', 'Estimating']\",\"question\":\"A kindergarten teacher has students determine the following. ‚Ä¢ How many students are absent that day ‚Ä¢ If there are enough snack cups for each student to have one during snack time ‚Ä¢ How many balls are taken outside during recess and how many are brought back inside after recess. Which of the following concepts is most closely aligned with the determinations the teacher is having the students make?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}