{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f757d889-f846-5cb5-9988-5e3d7ca52a6c","task_key":"default--test--2156","task_revision_id":"1","upstream_id":"2156","short_description":"A psychologist emphasizes that parents should use appropriate consequences when…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Adlerian', 'Existential', 'Cognitive-behavioral', 'Systems theory', 'Gestalt', 'Transactional analysis', 'Psychodynamic', 'Humanistic', 'Behavioral', 'Rogerian']\",\"question\":\"A psychologist emphasizes that parents should use appropriate consequences when dealing with their children’s misbehavior. In addition, the psychologist explains that the goal of misbehavior can be to confirm an assumed disability, or to get attention, power, or revenge. Which of the following theoretical orientations is being used\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}