{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28a8bed1-9516-5828-9da0-f7c931c6795e","task_key":"default--test--2507","task_revision_id":"1","upstream_id":"2507","short_description":"Psychologist’s 35-year-old patient admits to sexually molesting a nine-year-old…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"Inform the patient's family members about the situation before notifying the authorities\\\", 'Seek advice from a professional ethics committee before taking any action', 'Discuss the issue further in therapy without involving the authorities', 'Notify the local child protection services without informing the patient', 'notify the authorities immediately and inform the patient that this is being done', 'Encourage the patient to turn himself into the authorities voluntarily', 'warn the patient to cease the molestations immediately or risk loss of confidentiality and being reported to the authorities', 'handle the issue within the therapeutic situation and thereby maintain confidentiality', \\\"inform the child's parents, who are responsible for the child’s well-being, before notifying the authorities\\\", 'Keep the information confidential as it was shared in therapy']\",\"question\":\"Psychologist’s 35-year-old patient admits to sexually molesting a nine-year-old child in the neighborhood. This patient has prior history of such behavior. The psychologist should\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}