{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6b965c3-cb4a-56ea-ba3e-7c621804a073","task_key":"default--test--2121","task_revision_id":"1","upstream_id":"2121","short_description":"A major problem in thinking is maintaining alertness or concentration. What have…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The human mind can easily concentrate for long periods of time.', 'Concentration is unaffected by external distractions if the subject is engaging enough.', 'Attention can be sustained indefinitely if the subject is of personal interest.', 'The human system responds poorly to novelty or change in stimuli.', 'Frequent breaks diminish the ability to focus on tasks.', 'It is difficult to maintain attention for a long time on one particular subject or source of stimulation.', 'The human system shows increased alertness in response to monotonous stimuli.', 'Alertness is constant and does not fluctuate throughout the day.', 'The mind is better at multitasking than focusing on a single task for an extended period.', 'Concentration improves with time.']\",\"question\":\"A major problem in thinking is maintaining alertness or concentration. What have experimental findings demonstrated about this problem?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}