{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c02f4ed-c598-5bb6-8c17-172c8a8f56c9","task_key":"all--test--3c02f4ed-c598-5bb6-8c17-172c8a8f56c9","task_revision_id":"1","upstream_id":"","short_description":"Which of the following best states a conclusion of Stanley Milgram's study of…","config":"all","split":"test","body":"{\"choices\":\"['When faced with a difficult decision, people are more likely to follow the opinions of the crowd than to diverge from those opinions.', 'When challenged to commit a crime, people will most often refuse if they are not coerced, but will agree if an authority figure coerces them.', 'When asked to inflict pain on others, people are likely to inflict mild amounts of pain without remorse.', 'When an authority figure is present, people are more likely to obey orders than to question the wisdom of the orders.']\",\"question\":\"Which of the following best states a conclusion of Stanley Milgram's study of obedience?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}