{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0bc8e49d-10cc-5934-ab59-906f396ee7ac","task_key":"all--test--0bc8e49d-10cc-5934-ab59-906f396ee7ac","task_revision_id":"1","upstream_id":"","short_description":"Why did the spear-thrower represent a useful innovation and to when does it…","config":"all","split":"test","body":"{\"choices\":\"['It made warfare so deadly that it became obsolete beginning in the Middle Paleolithic and Middle Stone Age.', 'It allowed artistic expression by hunters beginning in the Middle Paleolithic and Middle Stone Age.', 'It launched larger spears more quickly and more frequently beginning in the Upper Paleolithic and Late Stone Age.', 'It launched projectiles with greater force and accuracy beginning in the Upper Paleolithic and Late Stone Age.']\",\"question\":\"Why did the spear-thrower represent a useful innovation and to when does it date?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}