{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3ee5e18-36c7-5237-98da-1d52ba5d965b","task_key":"default--test--5179","task_revision_id":"1","upstream_id":"5179","short_description":"A kindergarten teacher sets up a literacy center with an activity that asks…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Letter recognition', 'Phonemic awareness', 'Sentence construction', 'Word recognition', 'Reading comprehension', 'Spelling skills', 'Syllable counting', 'Vocabulary', 'Handwriting', 'Phonics']\",\"question\":\"A kindergarten teacher sets up a literacy center with an activity that asks students to match pictures of objects with pictures of letters to correctly indicate the beginning sound. Which of the following is the teacher trying to develop in students through the center?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}