{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f7e9c60-a1b6-5760-9e20-b6b9a2c21723","task_key":"default--test--2074","task_revision_id":"1","upstream_id":"2074","short_description":"A person in the pseudoindependent stage of White racial identity is currently…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Advocating for racial equality without acknowledging their own racial privilege', 'Convinced that racism is a problem of the past and no longer relevant', 'Developing an awareness of the role of Whites in perpetrating racism', 'Unaware of race and racism', 'Actively participating in racist activities', 'Fully accepting of their own racial biases without any desire to change', 'Attempting to resolve moral dilemmas associated with an awareness of race and racism', 'Exploring what it means to be White and confronting own biases', 'Trying to distance themselves from their White identity', 'Completely rejecting the concept of race and racism']\",\"question\":\"A person in the pseudoindependent stage of White racial identity is currently ___________.\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}