{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9731a140-fa66-547d-88b0-a35e48290fe4","task_key":"all--test--9731a140-fa66-547d-88b0-a35e48290fe4","task_revision_id":"1","upstream_id":"","short_description":"A 64-year-old woman comes to the physician because of a 5-month history of…","config":"all","split":"test","body":"{\"choices\":\"['Mesothelioma', 'Metastatic adenocarcinoma of the breast', 'Multiple endocrine neoplasia', 'Non-small cell lung carcinoma']\",\"question\":\"A 64-year-old woman comes to the physician because of a 5-month history of increasing shortness of breath, sore throat, and a cough productive of a small amount of white phlegm. Over the past week, she has had nausea related to excess coughing. Over the past year, she has had a 3.2-kg (7-lb) weight loss. She has asthma treated with theophylline and inhaled β-adrenergic agonists and corticosteroids. She has smoked one pack of cigarettes daily for 44 years and drinks one alcoholic beverage daily. She appears thin. Examination shows a 2-cm, nontender lymph node in the right supraclavicular area. Examination shows no other abnormalities. An x-ray of the chest shows a large right lower lobe density. A CT scan of the chest shows a 7.5 x 7.5 x 6-cm right lower lobe mass with some scattered calcifications. The lesion abuts the posterior chest wall without clear invasion. There are right lower peritracheal, precarinal, right hilar, and subcarinal lymph nodes. There is a 1.5-cm mass in the right adrenal gland. A biopsy specimen of the lung mass is most likely to show which of the following?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}