{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dfa4c54-dfc0-5f9f-9753-e63d842304e9","task_key":"all--test--4dfa4c54-dfc0-5f9f-9753-e63d842304e9","task_revision_id":"1","upstream_id":"","short_description":"In what ways might a society be threatened?","config":"all","split":"test","body":"{\"choices\":\"['Societies are threatened when their linguistic heritage and traditions are under attack. The inability to use and develop traditional language signifies the destruction of a society.', \\\"Threats to societal security exist when a society regards its collective identity as being under question. Such circumstances include the suppression of expression and interference with a society's ability to reproduce itself across generations.\\\", 'Military threats to societies in the form of external aggression, mainly through de-population policies with the intent of preventing collective identities from being reproduced over generations.', 'Migratory influxes into a host receiving state results in a shift in the composition of the population resulting in horizontal competition or vertical competition. This is not an outright threat to survival but part of wider uncontrollable processes of evolutionary identity.']\",\"question\":\"In what ways might a society be threatened?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}