{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80417ce4-94bf-53e4-a309-1566fe36fa44","task_key":"all--test--80417ce4-94bf-53e4-a309-1566fe36fa44","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The triumph of avant-garde literary styles over traditional realism', 'The adaptation of Western literary forms by non-Western authors', 'The political use of art and literature for propagandistic purposes', 'The response of interwar literary figures to the horrors of World War I']\",\"question\":\"This question refers to the following information.\\nNew York! I say New York, let black blood flow into your blood.\\nLet it wash the rust from your steel joints, like an oil of life\\nLet it give your bridges the curve of hips and supple vines…\\nSee your rivers stirring with musk alligators\\nAnd sea cows with mirage eyes…\\nJust open your eyes to the April rainbow\\nAnd your eyes, especially your ears, to God\\nWho in one burst of saxophone laughter\\nCreated heaven and earth in six days,\\nAnd on the seventh slept a deep Negro sleep.\\nSenegalese poet Léopold Sédar Senghor, \\\"To New York,\\\" 1956\\nThe verse above is best seen as belonging to which of the following modern cultural developments?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}