{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74eec612-dff0-5308-91c9-5b031b9dd5d6","task_key":"default--test--2340","task_revision_id":"1","upstream_id":"2340","short_description":"Longitudinal studies of children with specific reading disabilities show that","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['these children usually have a high level of social skills', 'these children usually have above-average IQ levels', 'the reading impairments are usually accompanied by a high vocabulary level', 'these children usually have subtle hearing disorders', 'these children usually excel in artistic activities', 'these children usually excel in sports', 'the reading impairments usually improve significantly with age', 'the reading impairments usually accompanied by a low vocabulary level', 'these children usually have exceptional mathematical skills', 'reading chills usually do not reach normal levels as these children reach adulthood']\",\"question\":\"Longitudinal studies of children with specific reading disabilities show that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}