{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c678ab7a-2dc2-5376-887d-a50f308fe99e","task_key":"default--test--2003","task_revision_id":"1","upstream_id":"2003","short_description":"You have been treating a teenage male you suspect may have some neurologic…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"Ask your client's parents to send the records\\\", 'Ask the neurologist to sign a confidentiality agreement before sending the records', 'Inform the neurologist that you would need a release of information signed by your client’s parents', 'Request that the neurologist obtain the records from the hospital', 'Send the records only if the parents give verbal consent', 'Refuse to send the records, citing patient confidentiality', 'Ask your client whether he wishes to have the records released to the neurologist', 'Tell the neurologist the parents need to request the records', 'Send a summary of your notes, but withhold the full records', 'Send the records as requested']\",\"question\":\"You have been treating a teenage male you suspect may have some neurologic difficulties. You inform his parents about your concerns and suggest they send him to a neurologist for an evaluation. Sometime later, the neurologist contacts you and requests your progress notes. You should ___________.\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}