{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d1e6b257-b424-5413-82ae-bd95dfd9d1ca","task_key":"default--test--2093","task_revision_id":"1","upstream_id":"2093","short_description":"The highest levels of learning and performance are usually associated with:","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['constant changes in levels of arousal.', 'low levels of arousal.', 'arousal levels that are consistently low.', 'high levels of arousal.', 'complete absence of arousal.', 'moderate levels of arousal.', 'extreme levels of arousal.', 'variability in levels of arousal.', 'gradually increasing levels of arousal.', 'arousal levels that are consistently high.']\",\"question\":\"The highest levels of learning and performance are usually associated with:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}