{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72bd6806-0ee3-52b9-a3fd-52af02817edb","task_key":"default--test--2031","task_revision_id":"1","upstream_id":"2031","short_description":"Before Justin could take an airplane flight, he needed to overcome his fear of…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['psychoanalysis', 'behavior therapy', 'cognitive therapy', 'humanistic therapy', 'existential therapy', 'rational emotive behavior therapy', 'mindfulness-based cognitive therapy', 'group therapy', 'gestalt therapy', 'reality therapy']\",\"question\":\"Before Justin could take an airplane flight, he needed to overcome his fear of flying. His therapist taught him relaxation exercises, which he practiced while first looking at pictures of airplanes, then seeing them take off at the airport, then going into an empty plane that would not take off, then finally taking a short flight. Which of the following treatments did he undergo?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}