{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89869240-a33b-57b2-b160-11575381e987","task_key":"default--test--2354","task_revision_id":"1","upstream_id":"2354","short_description":"Detection of a just-noticeable difference (JND) depends on the","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['presence of a \\\"no stimulus\\\" control and the sensitivity of the signal-detection equipment', 'initial determination of the absolute threshold and the variation of the difference threshold', 'frequency of the existing stimulus and the presence of one or more sensory modalities', 'intensity of the new stimulus and that of the stimulus already present']\",\"question\":\"Detection of a just-noticeable difference (JND) depends on the\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}