{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d13e7b7c-5fb4-5830-8de0-82e0c8edf3cf","task_key":"default--test--5747","task_revision_id":"1","upstream_id":"5747","short_description":"It's 3:00 A.M. You are male. You awaken to urinate. Standing at the commode you…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['gonorrhea', 'syphilis', 'chlamydia', 'urinary tract infection', 'penile cancer', 'prostate infection', 'genital herpes', 'kidney stones', 'bladder infection', 'AIDS']\",\"question\":\"It's 3:00 A.M. You are male. You awaken to urinate. Standing at the commode you are in extreme agony. Your penis feels as if it is on fire. You notice a yellowish discharge coming out of your penis after you finish urinating. Your moans have awakened your significant other. She comes running into the bathroom and asks you what is wrong. You tell her you probably have:\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}