{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e3a6013e-60b2-50de-bbb9-b44b2c58c81a","task_key":"all--test--e3a6013e-60b2-50de-bbb9-b44b2c58c81a","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Slavery', 'Indentured servitude', 'The encomienda system', 'The apprentice system']\",\"question\":\"This question refers to the following information.\\nThe invaders had brought in wheat and other Eurasian and African grains; peach, pear, orange, and lemon trees; chick-peas, grape vines, melons, onions, radishes, and much more. A Spanish nobleman come to America could require his Indians to furnish his table with the fruits of his ancestors.\\n—Alfred Crosby, historian, 1972\\nWhat economic practice is referred to by the phrase \\\"his Indians\\\" in the quote?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}