{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d97269cf-4121-520b-b7fe-e1d6488e5ce7","task_key":"default--test--6007","task_revision_id":"1","upstream_id":"6007","short_description":"Which of the following releases most energy when completely oxidised in the…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['One gram of glycerol', 'One gram of lactose', 'One gram of fructose', 'One gram of palmitic acid', 'One gram of cellulose', 'One gram of glucose', 'One gram of alcohol', 'One gram of leucine']\",\"question\":\"Which of the following releases most energy when completely oxidised in the body?\",\"src\":\"ori_mmlu-college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}