{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aea9bf32-2faa-5fe2-8ee1-b7e261e8183d","task_key":"default--test--6467","task_revision_id":"1","upstream_id":"6467","short_description":"Which of the following is true about rectal bleeding?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Hemorrhoids are never a cause of rectal bleeding', 'A painful digital examination suggests an anal fissure', 'A colonoscopy is not necessary if rectal bleeding is present', 'Rectal cancer is the commonest cause of rectal bleeding', 'Bright red blood on the finger usually signifies a caecal lesion', 'Rectal bleeding is always a sign of cancer', 'Proctoscopy is of no value in the assessment of rectal bleeding', 'Rectal bleeding is not a symptom of diverticulosis', 'Rectal bleeding is always accompanied by pain', 'A negative colonoscopy rules out all causes of rectal bleeding']\",\"question\":\"Which of the following is true about rectal bleeding?\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}