{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0721a7bc-e84c-5ac0-a715-5c7651a4a9ba","task_key":"default--test--2451","task_revision_id":"1","upstream_id":"2451","short_description":"One of the most frequent findings of studies on human beings with “split brains”…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The right hemisphere is predominantly concerned with verbal information', 'The right hemisphere is predominantly concerned with numeric information', 'aside from the long-known localization of speech functions, the two hemispheres appear to be essentially equivalent as information processors', 'certain kinds of visuospatial tasks are performed beer by the right hemisphere than by the left', 'The two hemispheres appear to have a competitive relationship in information processing', 'the right hemisphere provides backup for the left in some tasks but excels the left in none', \\\"The left hemisphere is predominantly concerned with handling the body's motor skills\\\", 'The left hemisphere provides backup for the right in some tasks but excels the right in none', 'The left hemisphere predominantly handles emotional processing', 'Both hemispheres are specialized in processing musical information']\",\"question\":\"One of the most frequent findings of studies on human beings with “split brains” is that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}