{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8cabae0e-5f39-577f-b249-33938ca28540","task_key":"default--test--2080","task_revision_id":"1","upstream_id":"2080","short_description":"Dr. Marcos Manzetti uses a “sliding scale” that is based on a client’s current…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['not mentioned in the ethical guidelines.', 'ethically acceptable, as it provides opportunities for those with lower income.', 'explicitly recommended in the ethical guidelines.', 'ethically acceptable since it serves the best interests of his clients.', 'explicitly recommended in the legal guidelines.', 'ethically unacceptable, as it can be considered discriminatory.', \\\"ethically questionable, as it may not consider a client's overall financial situation.\\\", 'explicitly prohibited in the ethical guidelines.', 'a common practice, but not explicitly mentioned in the ethical guidelines.', 'ethically unacceptable since it treats clients inequitably.']\",\"question\":\"Dr. Marcos Manzetti uses a “sliding scale” that is based on a client’s current income to set his fees for therapy. Dr. Manzetti’s practice is:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}