{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfebdb23-79d7-5d6d-af12-a7ef4f0de7ba","task_key":"all--test--bfebdb23-79d7-5d6d-af12-a7ef4f0de7ba","task_revision_id":"1","upstream_id":"","short_description":"A new highway extends from a major city though an area used for farmland. Which…","config":"all","split":"test","body":"{\"choices\":\"['Farmers will increase the acreage under production.', 'The city will evolve into a multiple nuclei model.', 'Suburbanization will develop because the time it takes to commute to the city has been decreased.', 'People from the countryside will move to the city, depopulating the farmland region.']\",\"question\":\"A new highway extends from a major city though an area used for farmland. Which of the following is most likely to happen as a result?\",\"subject\":\"sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}