{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c959400e-dff1-5c66-a3f8-6aa47fddfa66","task_key":"default--test--6047","task_revision_id":"1","upstream_id":"6047","short_description":"A 65-year-old woman presents with progressive muscle weakness that worsens…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Presynaptic neuromuscular junction', 'Postsynaptic neuromuscular junction', 'Motor cortex', 'Peripheral nerves', 'Muscle fibers', 'Spinal cord', 'Basal ganglia']\",\"question\":\"A 65-year-old woman presents with progressive muscle weakness that worsens throughout the day. She reports difficulty climbing stairs, brushing her hair, and speaking for extended periods. Symptoms improve with rest. She has recently experienced double vision and drooping eyelids, especially in the evening. Physical examination shows fatigable weakness of proximal muscles and ptosis. She has no sensory deficits. CT scan of the chest reveals a mass in the anterior mediastinum. Which of the following is the most likely anatomical site of the underlying pathology?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}