{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"720541c0-7068-5629-bacc-acacc9ef5400","task_key":"all--test--720541c0-7068-5629-bacc-acacc9ef5400","task_revision_id":"1","upstream_id":"","short_description":"A 52-year-old female presents to the office with the complaint of greenish…","config":"all","split":"test","body":"{\"choices\":\"['breast cancer', 'fibrocystic disease', 'mammary duct ectasia', 'pituitary adenoma']\",\"question\":\"A 52-year-old female presents to the office with the complaint of greenish discharge from her left nipple. There is no history of trauma or fever, and the patient is not currently using oral contraceptives nor is she on hormone replacement therapy. Her last pregnancy was approximately 20 years ago. Physical examination findings are normal except for the green discharge. The most likely cause of this patient's presenting complaint is\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}