{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c221bf2-7905-5b18-a23b-cc508e4c8b4d","task_key":"default--test--2519","task_revision_id":"1","upstream_id":"2519","short_description":"Babbling ordinarily begins at about 4 to 5 months of age and initially includes:","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['phonemes from the child’s native language only.', 'Phonemes from the language most frequently spoken around the child.', \\\"Phonetic sounds specific to the child's geographical location.\\\", \\\"Phonemes from the parents' native language only.\\\", 'morphemes that the child has heard most frequently.', 'Phonetic sounds that are easiest for the child to produce.', 'morphemes from all languages.', \\\"Morphemes from the child's native language only.\\\", 'Words that the child has heard most frequently.', 'phonemes from all languages.']\",\"question\":\"Babbling ordinarily begins at about 4 to 5 months of age and initially includes:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}