{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e12a6ea2-d9c2-5219-b208-8bedd1f15289","task_key":"default--test--2436","task_revision_id":"1","upstream_id":"2436","short_description":"It is characteristic of brief crisis-oriented therapy to","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['avoid setting any specific goals to maintain flexibility', 'focus on developing a deep understanding of the crisis before setting goals', 'try to ensure that the client becomes calm and relaxed', 'agree upon clear-cut goals and keep the sessions focused on these goals', 'primarily use medication as a means of managing the crisis', 'encourage clients to disregard their issues until a later time', 'focus on building long-term coping mechanisms and strategies', 'rely on group therapy sessions to resolve the crisis', 'explore early childhood experiences related to the problem', 'be open and nondirective, allowing the client the greatest latitude in exploring all aspects of the crisis situation']\",\"question\":\"It is characteristic of brief crisis-oriented therapy to\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}