{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77ca2acb-57b9-5942-8b4d-ddd7132159b4","task_key":"default--test--2389","task_revision_id":"1","upstream_id":"2389","short_description":"As a child, Fara’s responsibilities were to help her mother in the kitchen while…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Self-Concept Therapy', 'Feminist Object Relations', 'Household Responsibilities', 'Maternal Role Therapy', 'Empowerment Therapy', 'Family Dynamics Therapy', 'Child Responsibility Therapy', \\\"Women's Rights Therapy\\\"]\",\"question\":\"As a child, Fara’s responsibilities were to help her mother in the kitchen while her brother and father mowed the lawn. She was expected to do the housework like her mother while her brother did yard work like their father. What part of Feminist Therapy addresses these issues?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}