{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d640f3fa-9dc4-57e5-9e42-44a51c48732c","task_key":"all--test--d640f3fa-9dc4-57e5-9e42-44a51c48732c","task_revision_id":"1","upstream_id":"","short_description":"In the definition of a public good, the term \"nonrival\" means that","config":"all","split":"test","body":"{\"choices\":\"[\\\"one person's use of the public good does not reduce the amount available to others\\\", 'the public good is produced by the government without any competition', 'the public good is produced at zero marginal cost', 'the public good is mostly consumed by free riders']\",\"question\":\"In the definition of a public good, the term \\\"nonrival\\\" means that\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}