{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d101e9b9-d455-591b-9eb8-5b9187dfc9cf","task_key":"all--test--d101e9b9-d455-591b-9eb8-5b9187dfc9cf","task_revision_id":"1","upstream_id":"","short_description":"What is the relationship between HM and realism?","config":"all","split":"test","body":"{\"choices\":\"['Both HM and realism are based on the notion that there exist within society certain discernable realities.', 'HM has some aspects in common with realism, liberalism, and social constructivism, but it offers something distinctive in having class conflict within capitalism as its central focus.', 'Whilst HM recognises the importance of the state as an actor within the international system, it also purports that states prioritize the interests of the ruling-class elite.', 'All of the above']\",\"question\":\"What is the relationship between HM and realism?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}