{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98e62f83-92c2-5b18-9fb1-9e74f2d819c5","task_key":"default--test--2592","task_revision_id":"1","upstream_id":"2592","short_description":"A psychologist focusing on whether development occurs in stages is most…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['rationality versus irrationality', 'stability versus change', 'cognition versus behavior', 'qualitative versus quantitative data', 'conscious versus unconscious influences', 'nature versus nurture', 'subjectivity versus objectivity', 'individual versus societal influence', 'free will versus determinism']\",\"question\":\"A psychologist focusing on whether development occurs in stages is most interested in which of the following controversies?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}