{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c02a642-6db1-52cb-8ac5-f82809237725","task_key":"all--test--0c02a642-6db1-52cb-8ac5-f82809237725","task_revision_id":"1","upstream_id":"","short_description":"Why do scientists think Mars was once warmer and wetter?","config":"all","split":"test","body":"{\"choices\":\"['Mars shows evidence of significant volcanism which implies that the climate was once warmer and therefore wetter', 'early observations showed what appeared to be changing vegetation patterns and canals indicating the presence of water', 'there are many geologic features on Mars that are difficult to explain unless liquid water was once stable at the surface', 'all planets tend to begin with warm wet climates and gradually become cold and dry with smaller planets cooling faster than larger planets']\",\"question\":\"Why do scientists think Mars was once warmer and wetter?\",\"subject\":\"astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}