{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"101a4083-ac7e-5f91-a801-ac076320be4b","task_key":"default--test--6590","task_revision_id":"1","upstream_id":"6590","short_description":"Which products of ADP degradation increase in concentration in the blood during…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Ammonia, urea and creatine.', 'Ammonia, hypoxanthine and creatinine.', 'Ammonia, creatine and uric acid.', 'Urea, creatine and uric acid.', 'Ammonia, urea and uric acid.', 'Urea, creatinine and uric acid.', 'Ammonia, urea and creatinine.', 'Urea, creatinine and creatine.', 'Urea, hypoxanthine and uric acid.', 'Ammonia, hypoxanthine and uric acid.']\",\"question\":\"Which products of ADP degradation increase in concentration in the blood during multiple sprint sports?\",\"src\":\"ori_mmlu-college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}