{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51e94b96-9fe8-5b99-8412-eeff4e9ab889","task_key":"default--test--5299","task_revision_id":"1","upstream_id":"5299","short_description":"Freud's notion of the 'ego' referred to:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the self as a whole: an unstable mix of conscious and unconscious elements', 'the part of the mind that deals with primal instincts and survival', 'the neurotic part of the mind that longs for belonging and may suffer an inferiority complex', 'the part of the mind that handles social and cultural expectations', 'the unconscious mass of instinctive drives that may be repressed', 'the part of the mind that is responsible for dream interpretation', 'the unconscious part of the mind that stores memories and experiences', 'the conscious part of the mind that regulates emotional drives on a practical, rational level', 'the conscious part of the mind that manages rational thought and problem-solving', 'the part of the mind that mediates between the id and the superego']\",\"question\":\"Freud's notion of the 'ego' referred to:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}