{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68e0e874-0219-51df-ac51-265544cdb454","task_key":"default--test--6635","task_revision_id":"1","upstream_id":"6635","short_description":"The function of the gastrointestinal tract can be classified in several ways.…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Extraction of maximum value from nutrients', 'Production of digestive enzymes', 'Stimulation of appetite', 'Processing of food into a liquid form', 'Regulation of body temperature', 'Absorption and retention of water and electrolytes', 'Maintenance of barrier function', 'Control of blood sugar levels', 'Elimination of toxins', 'Production of hormones for growth and repair']\",\"question\":\"The function of the gastrointestinal tract can be classified in several ways. Which of the following functions is most important for survival immediately after a meal?\\n \",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}