{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4660564-c031-540e-9978-128d72b7f73c","task_key":"all--test--e4660564-c031-540e-9978-128d72b7f73c","task_revision_id":"1","upstream_id":"","short_description":"A 24-year-old man comes to the office because of a 2-day history of a red, itchy…","config":"all","split":"test","body":"{\"choices\":\"['Apocrine gland', 'Dermis', 'Eccrine gland', 'Hair follicle']\",\"question\":\"A 24-year-old man comes to the office because of a 2-day history of a red, itchy rash on his buttocks and legs. Four days ago, he returned from a cruise to the Caribbean, during which he swam in the ship’s pool and used the hot tub. He appears well. His vital signs are within normal limits. Physical examination shows the findings in the photograph. The infectious agent causing these findings most likely began to proliferate in which of the following locations?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}