{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af82cd1b-0cdb-5997-8b50-8f781625193d","task_key":"all--test--af82cd1b-0cdb-5997-8b50-8f781625193d","task_revision_id":"1","upstream_id":"","short_description":"The following statements define the conceptualization of 'class' within HM;…","config":"all","split":"test","body":"{\"choices\":\"['All advocates of HM accept that social class is important.', 'Within the HM paradigm, the only form of class that matters is social class.', 'The focus of HM lies predominantly on class and class in relation to capital.', 'HM regards class as being far more complex than a simple division between a capitalist ruling class and working class.']\",\"question\":\"The following statements define the conceptualization of 'class' within HM; which is incorrect?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}