{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a632fb8-6fb9-5de3-9ecd-5746b035a3c7","task_key":"default--test--6255","task_revision_id":"1","upstream_id":"6255","short_description":"A 30-year-old woman comes to the physician because of a 2-day history of…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Both CFTR alleles must be mutated to produce symptoms', 'The CFTR mutation is associated with her clinical phenotype but not the cause', \\\"The patient's CFTR mutation is unrelated to her clinical phenotype\\\", \\\"The patient's symptoms are coincidental and not related to the CFTR mutation\\\", 'The patient has a rare form of CFTR mutation that requires two alleles', 'The G551D mutation in CFTR does not cause symptoms', \\\"The patient's clinical phenotype is due to a mutation in a different gene\\\", 'The patient is a CFTR obligate carrier', 'Only one G551D allele is needed in CFTR', 'The second CFTR mutation was not detected by the testing obtained']\",\"question\":\"A 30-year-old woman comes to the physician because of a 2-day history of abdominal pain. She has a history of recurrent upper respiratory tract infections, sinusitis, and pancreatitis. She has thick nasal secretions. She says that her sweat is salty and crystallizes on her skin. Her vital signs are within normal limits. Physical examination shows epigastric tenderness. Genetic testing for the 36 most common mutations shows a detectable mutation (G551D) in one allele of the CFTR gene. Which of the following best explains this patient's clinical phenotype? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}