{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b5989a8-336b-5345-8db6-dd57f5cd1ab8","task_key":"default--test--2245","task_revision_id":"1","upstream_id":"2245","short_description":"Discuss three ways in which concepts can be attained.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Through the attachment of labels to entities, through the use of contexts, and through the use of definitions', 'Through comparison, through contrast, and through categorization', 'Through reading, through writing, and through speaking', 'Through repetition, through memorization, and through association', 'Through intuition, through dreaming, and through meditation', 'Through observation, through experimentation, and through inference', 'Through listening, through discussion, and through critical thinking', 'Through sensory experience, through imitation, and through reflection', 'Through trial and error, through practice, and through feedback', 'Through visualization, through problem-solving, and through storytelling']\",\"question\":\"Discuss three ways in which concepts can be attained.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}