{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1214a1ea-7d47-5437-af92-775803a1921b","task_key":"all--test--1214a1ea-7d47-5437-af92-775803a1921b","task_revision_id":"1","upstream_id":"","short_description":"Donny threw a knife at Sally, intending to injure her severely. However, Donny…","config":"all","split":"test","body":"{\"choices\":\"['Donny will be liable for battery, but not assault.', 'Donny will be liable for assault, but not battery.', 'Donny will be liable for assault and for battery.', 'Donny will not be liable for either assault or battery because this is only a criminal matter.']\",\"question\":\"Donny threw a knife at Sally, intending to injure her severely. However, Donny missed Sally. Sally saw the knife just as it whizzed by her head, missing it by about one inch. As a result, Sally was very scared. Sally sued Donny for assault and battery. Which of the following is most correct?\",\"subject\":\"jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}