{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce741b34-88e6-5721-be97-bf8683fd8630","task_key":"all--test--ce741b34-88e6-5721-be97-bf8683fd8630","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"[\\\"In ancient Rome, religious worship was decentralized and tended to vary with one's social position.\\\", 'In ancient Rome, religious worship was the source of much social tension and turmoil.', 'In ancient Rome, religious worship was homogeneous and highly centralized.', 'In ancient Rome, religious worship was revolutionized by the introduction of Christianity.']\",\"question\":\"This question refers to the following information.\\nRead the the following quotation to answer questions.\\nThe various modes of worship which prevailed in the Roman world were all considered by the people as equally true; by the philosopher as equally false; and by the magistrate as equally useful.\\nEdward Gibbon, The Decline and Fall of the Roman Empire, 1776–1788\\nGibbon's interpretation of the state of religious worship in ancient Rome could be summarized as\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}