{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2936bcb1-b9f8-5807-9eba-5e510bb7626f","task_key":"default--test--2455","task_revision_id":"1","upstream_id":"2455","short_description":"Object relations therapy is similar to Freudian psychodynamic therapy in its…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the neutral role of the therapist', \\\"the client's interpersonal relationships in adulthood\\\", 'early childhood experiences', \\\"the client's conscious thoughts and feelings\\\", 'the use of cognitive behavioral strategies', 'issues of attachment and separation', 'the ego, id, and superego', 'the impact of social factors on mental health', \\\"the role of the family system in the client's mental health\\\", 'the exploration of dreams and their meanings']\",\"question\":\"Object relations therapy is similar to Freudian psychodynamic therapy in its primary focus on\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}