{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"33f5b346-f6b0-5900-8305-d6bb600103ec","task_key":"all--test--33f5b346-f6b0-5900-8305-d6bb600103ec","task_revision_id":"1","upstream_id":"","short_description":"Why did Ptolemy have the planets orbiting Earth on ‘circles upon circles’ in his…","config":"all","split":"test","body":"{\"choices\":\"['To explain why more distant planets take longer to make a circuit through the constellations of the zodiac.', 'To explain the fact that planets sometimes appear to move westward rather than eastward relative to the stars in our sky.', 'To explain why the Greeks were unable to detect stellar paralax.', 'To properly account for the varying distances of the planets from Earth.']\",\"question\":\"Why did Ptolemy have the planets orbiting Earth on ‘circles upon circles’ in his model of the universe?\",\"subject\":\"astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}