{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4c163bb-9727-58d7-9682-bee9de6fc4f3","task_key":"default--test--2541","task_revision_id":"1","upstream_id":"2541","short_description":"What are the two major functions of the spinal cord? Discuss.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Stores memories and processes complex thought', 'Acts as a messenger and directs some simple actions independently', 'Controls breathing and circulates lymph', 'Regulates hormone release and immune response', 'Transmits sensory information and controls voluntary movements', 'Coordinates digestive processes and metabolic activity', 'Filters blood and produces red blood cells', 'Conducts sound waves and assists in balance', 'Regulates body temperature and controls heart rate', 'Enables communication between the brain and the rest of the body']\",\"question\":\"What are the two major functions of the spinal cord? Discuss.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}