{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"2cb8aaab-afb5-572c-97e3-de2e46a4f8eb","task_key":"all--test--2cb8aaab-afb5-572c-97e3-de2e46a4f8eb","task_revision_id":"1","upstream_id":"","short_description":"The crown of a very carious tooth suddenly fractures while the patient is…","config":"all","split":"test","body":"{\"choices\":\"['right lung because the right main bronchus is wider and more vertical than the left.', 'left lung because the left main bronchus is wider and more vertical than the right.', 'right lung or left lung because the two main bronchi are of equal size and at the same angulation.', 'right or left lung because there are no factors predisposing to the fragment going in one particular direction.']\",\"question\":\"The crown of a very carious tooth suddenly fractures while the patient is standing eating at a take-away food outlet and a fragment is inhaled. It is more likely to enter the\",\"subject\":\"anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}