{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"95b7fd24-eeac-570d-af7b-ae0a86e72f4b","task_key":"default--test--6185","task_revision_id":"1","upstream_id":"6185","short_description":"A 45-year-old man comes to the physician because of right shoulder pain that…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Deltoid', 'Infraspinatus', 'Supraspinatus', 'Triceps Brachii', 'Subscapularis', 'Coracobrachialis', 'Teres Major', 'Pectoralis', 'Brachialis', 'Biceps Brachii']\",\"question\":\"A 45-year-old man comes to the physician because of right shoulder pain that began after he chopped wood 2 days ago. Examination of the right upper extremity shows no obvious bone deformities or point tenderness. The pain is reproduced when the patient is asked to externally rotate the shoulder against resistance; there is no weakness. In addition to the teres minor, inflammation of which of the following tendons is most likely in this patient?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}