{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a662c8fb-47bd-5eb3-a9de-8a49d6fd0340","task_key":"default--test--2140","task_revision_id":"1","upstream_id":"2140","short_description":"At the outset of group counseling, the therapist should explain char the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['on the “honor system” only', 'protected by state/provincial laws', 'less of an issue than it is in individual therapy', 'mandated by federal law', 'the legal responsibility of the therapist', 'only applicable to the therapist', 'only necessary if sensitive information is shared', \\\"dependent on the group's consensus\\\", 'not applicable in group settings']\",\"question\":\"At the outset of group counseling, the therapist should explain char the obligation of the group member to respect confidentiality is\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}