{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0e53cdc-3ff3-5aba-825d-229c7bb42701","task_key":"default--test--6035","task_revision_id":"1","upstream_id":"6035","short_description":"A 29-year-old male presents to the office with the complaints of a pounding…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['social anxiety disorder', 'adjustment disorder with anxious mood', 'post-traumatic stress disorder', 'panic disorder', 'dysthymia', 'obsessive-compulsive personality disorder', 'major depressive disorder, single episode, moderate', 'bipolar disorder', 'generalized anxiety disorder', 'schizophrenia']\",\"question\":\"A 29-year-old male presents to the office with the complaints of a pounding heart, poor sleep, and a generalized feeling of restlessness. He relocated 2 months ago to take a promotion and has found himself unable to concentrate on his new duties. He worries about his ability to adapt to his increase in responsibility at work and states that he has called in sick 3 days within the past 2 weeks. The most likely diagnosis is\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}