{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e54efa3-4aee-5d23-b426-6c43f2bf7cec","task_key":"default--test--2075","task_revision_id":"1","upstream_id":"2075","short_description":"During the latter part of their first year, children’s language- learning…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['begin to utter words in an unsystematic and random manner', 'start to form full sentences', 'begin producing more language than they can comprehend', 'begin to utter: their first words in a systematic progression determined by selective reinforcement of babbling', 'lose the ability to understand words in their own language', 'begin to be sensitive to rhythms in speech patterns', 'lose the ability to perceive any sound distinctions', 'are able to perceive sound distinctions in multiple languages', 'become less able to perceive sound distinctions not made in their own language']\",\"question\":\"During the latter part of their first year, children’s language- learning ability changes in thar they\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}