{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdc83ba0-9008-5ff9-b66f-8684f0a9b4dd","task_key":"all--test--bdc83ba0-9008-5ff9-b66f-8684f0a9b4dd","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"[\\\"Women's suffrage should increase\\\", 'Democratization of society', 'Increased disillusionment and cynicism', 'The rise of communism as a world power']\",\"question\":\"This question refers to the following information.\\nAddison's Act had been conceived in the belief that, unless working-class aspirations were quickly met after the war, Britain might experience a revolution similar to that in Russia. By 1920 that fear was beginning to recede and Addison's policy was being regarded as extravagant. An immediate victim of the new attitude was the Borough's Kingfield Street scheme, provisional plans for which were approved in September 1920. Much to the Council's surprise, the Government's Housing Board deferred the scheme, \\\"having regard to the Council's present commitments and the money available at the present time, and in May 1921 the Government announced a drastic curtailment of the housing programme, cutting the housing target by half.\\n—Excerpt from a work by English historian Hermione Hobhouse Public Housing in Poplar: The Inter-war Years, 1994\\nWhich of the following attitudes prevalent after the First World War is evident in the passage above?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}