{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2fa8f52-b396-580f-9f97-8453f7ae81c5","task_key":"default--test--2428","task_revision_id":"1","upstream_id":"2428","short_description":"What are the shortcomings of clinical andscientific evaluationsin producing…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Ineffectiveness of therapies', 'Lack of trained professionals', 'Lack of valid, consistent criteria for evaluating improvement, difficulties in measuring the degree of improvement, and mismatch between the therapy and the specific type of patient', 'Insufficient focus on prevention and early intervention strategies', 'Overemphasis on pharmaceutical interventions rather than talking therapies', 'High cost of therapies']\",\"question\":\"What are the shortcomings of clinical andscientific evaluationsin producing information about the nature of psychotherapy ?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}