{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8c66a8b-091f-56a8-825c-9060e3e8fd94","task_key":"default--test--2114","task_revision_id":"1","upstream_id":"2114","short_description":"What factors within the individual are the basis for being able to either…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Socioeconomic status and personal beliefs', 'Childhood upbringing and current employment status', 'Diet and exercise habits', 'Personality type and coping mechanisms', 'Intelligence quotient (IQ) and gender', 'Marital status and number of children', \\\"Person's physical health and age\\\", 'Cultural background and language proficiency', 'Genetic predisposition and education level', 'Amount of conflict a person can handle and life experiences']\",\"question\":\"What factors within the individual are the basis for being able to either withstand conflict or develop neuroses?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}