{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"542fe1b8-fe92-5159-91d7-6ae4d7a57e3e","task_key":"default--test--2724","task_revision_id":"1","upstream_id":"2724","short_description":"According to Erikson, those who look back at the end of their lives with regrets…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['trust versus mistrust', 'intimacy versus isolation', 'ego identity versus role confusion', 'integrity versus despair', 'identity versus role confusion', 'generativity versus stagnation', 'industry versus inferiority', 'initiative versus guilt', 'isolation versus connection', 'autonomy versus shame and doubt']\",\"question\":\"According to Erikson, those who look back at the end of their lives with regrets and the feeling that their lives have lacked fulfillment evidence unsuccessful resolution of the challenge of\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}