{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b242e2f-426c-578e-a872-bd968e44ad72","task_key":"default--test--2707","task_revision_id":"1","upstream_id":"2707","short_description":"After 6 months of once-2-week therapy and several attempts to evaluate and…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Recommend the patient to join a support group', 'discuss the possibility that the patient consult with another therapist', 'Ask the patient to continue therapy for another 6 months before evaluating again', \\\"Ignore the patient's complaints and continue with the current treatment plan\\\", 'increase the frequency of sessions', 'Begin a new therapy approach without discussing with the patient', 'treat this as an instance of resistance', 'Suggest the patient to take a break from therapy', 'Prescribe medication to the patient', 'immediately terminate treatment and refer the patient elsewhere']\",\"question\":\"After 6 months of once-2-week therapy and several attempts to evaluate and adjust treatment goals and strategies, a patient continues to complain over several sessions thar therapy has not helped and that therapy goals have not been met. The psychologist should\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}