{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"15b797ab-cbca-5a31-aeb3-6c7a468e6841","task_key":"default--test--2616","task_revision_id":"1","upstream_id":"2616","short_description":"A psychologist who already has an established therapeutic relationship with a…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['agree to testify only if the court appoints her as an expert witness.', 'decline the request to testify.', \\\"agree to testify only if she feels it will benefit the child's mental health.\\\", 'agree to testify only if she obtains consent from both parents.', \\\"agree to testify only if she can remain the child's therapist.\\\", 'agree to testify without any conditions.', 'decline the request to testify unless she is given full access to the court proceedings.', 'agree to testify only after conducting a separate impartial evaluation.', 'agree to testify only if she can evaluate all of the involved parties.', \\\"agree to testify only if her familiarity with the child won''t bias her testimony.\\\"]\",\"question\":\"A psychologist who already has an established therapeutic relationship with a child is asked to act as an expert witness in a divorce proceeding involving custody of the child. The psychologist's best course of action would be to:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}