{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10303600-a055-52aa-9ff0-d959d9266a47","task_key":"all--test--10303600-a055-52aa-9ff0-d959d9266a47","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is not one of the unifying principles of Critical…","config":"all","split":"test","body":"{\"choices\":\"['Principle actors are social constructs that are made through political practices.', 'Natural-science methodology is the best way to study social science, interpretive methodologies lack the coherence required for in-depth analysis.', 'Knowledge of the social world is not objective, as there is no divide between the social world and knowledge of that world.', 'The purpose of theory is not explanation in terms of generalizable causal claims, but contextual understanding and practical knowledge.']\",\"question\":\"Which of the following is not one of the unifying principles of Critical Security Studies, as defined by Krause?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}