{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"093d7f52-f98f-51dd-b749-9e486a314bce","task_key":"all--test--093d7f52-f98f-51dd-b749-9e486a314bce","task_revision_id":"1","upstream_id":"","short_description":"Howard's idea of the garden city consisted of:","config":"all","split":"test","body":"{\"choices\":\"['six small towns surrounded by green belts and linked to a larger central city', 'planning cities to have plenty of trees, flowers and public gardens', 'each privately owned house to have its own front and back garden', 'tower blocks to displace the population vertically and leave most of the land available as green open spaces']\",\"question\":\"Howard's idea of the garden city consisted of:\",\"subject\":\"sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}