{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b27bbc82-ba9e-515a-abaf-08f168421b6d","task_key":"default--test--2582","task_revision_id":"1","upstream_id":"2582","short_description":"A correct statement about the early babbling of infants from different language…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['is different for each language community from the onset of babbling', 'depends upon the semantic structure of the language', 'depends upon the phonetic structure of the language', 'is the same for each language community until the first sentences are formed', 'is influenced by the musicality of the language spoken in the community', \\\"varies depending on the infant's exposure to multiple languages\\\", 'contains the full range of phonemes that comprise al languages', \\\"includes only the phonemes found in the language of the infant's community\\\", 'is the same for each language community until the first words are spoken', 'is different for each language community once the first words are spoken']\",\"question\":\"A correct statement about the early babbling of infants from different language communities is chat it\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}