{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5f5e6b3-abc7-53e1-a2d3-397475bf7cf2","task_key":"all--test--f5f5e6b3-abc7-53e1-a2d3-397475bf7cf2","task_revision_id":"1","upstream_id":"","short_description":"How were the royal cemeteries of the Moche discovered?","config":"all","split":"test","body":"{\"choices\":\"['Thieves raiding the tombs were caught by the police.', 'A forest fire burned away the dense vegetation that concealed the cemetery.', 'Archaeologists discovered an exact replica of the city in the map room.', 'A flock of sheep that had been living there fell into a tomb.']\",\"question\":\"How were the royal cemeteries of the Moche discovered?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}