{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b118009-0a31-5066-98c5-d393c6b68a0a","task_key":"all--test--7b118009-0a31-5066-98c5-d393c6b68a0a","task_revision_id":"1","upstream_id":"","short_description":"Mars has an atmosphere that is almost entirely carbon dioxide.Why isn't there a…","config":"all","split":"test","body":"{\"choices\":\"['Mars does not have enough internal heat to drive the greenhouse effect', 'Mars is too far from the sun for the greenhouse effect to work', 'the greenhouse effect requires an ozone layer which Mars does not have', 'the atmosphere on Mars is too thin to trap a significant amount of heat']\",\"question\":\"Mars has an atmosphere that is almost entirely carbon dioxide.Why isn't there a strong greenhouse effect keeping the planet warm?\",\"subject\":\"astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}