{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"795f384f-a534-5134-9af1-69d6bb70e320","task_key":"default--test--2649","task_revision_id":"1","upstream_id":"2649","short_description":"Atkinson, Morten, and Sue’s (1993) Racial/Cultural Identity Development Model…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['resistance and immersion', 'integrative awareness', 'immersion', 'reintegration', 'dissonance', 'encounter', 'pre-encounter', 'conformity', 'introspection', 'internalization and commitment']\",\"question\":\"Atkinson, Morten, and Sue’s (1993) Racial/Cultural Identity Development Model proposes that people in which of the following stages begin to question their rejection of all aspects of the dominant culture and absolute loyalty to their own culture?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}