{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d72cb49-1b5f-55f8-bcad-9d33ffdcbcaa","task_key":"default--test--2538","task_revision_id":"1","upstream_id":"2538","short_description":"Behavioral therapy typically alters the patterns of responding of clients by","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['influencing patients by controlling the consequences of their actions', 'encouraging clients to focus on their dreams and unconscious mind', \\\"promoting clients' insight into their own thoughts and feelings\\\", 'administering medication to manage symptoms', 'assisting clients to develop a positive self-image', \\\"vigorously challenging clients' illogical ways of thinking\\\", 'supporting clients in exploring their spiritual beliefs and practices', 'repeating or rephrasing what a client says during the course of therapy', 'encouraging clients to talk about their past experiences and traumas']\",\"question\":\"Behavioral therapy typically alters the patterns of responding of clients by\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}