{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80c7aff3-4e49-5f7e-8b08-61c4b59641e5","task_key":"default--test--6238","task_revision_id":"1","upstream_id":"6238","short_description":"A 54-year-old woman comes to the emergency department because of severe jaw pain…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Increased plasma protein-binding of codeine', 'Interference from the dental procedure', 'Insufficient dosage of codeine prescribed by the dentist', 'Allergic reaction to codeine preventing pain relief', 'Resistance to codeine developed over time', 'Decreased metabolism of codeine to morphine', 'Overuse of codeine leading to tolerance', 'Deficiency of κ receptors', 'Decreased absorption of codeine']\",\"question\":\"A 54-year-old woman comes to the emergency department because of severe jaw pain since undergoing a painful dental procedure 1 day ago. The patient was prescribed codeine after the procedure and instructed to take the medication every 4 hours, but she has continued pain despite adherence to this analgesic regimen. Other members of her family also have experienced poor pain control with codeine. Which of the following is the most likely explanation for this therapeutic failure?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}