{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"efef717c-cf89-59a4-ba1d-91d3fe32e324","task_key":"all--test--efef717c-cf89-59a4-ba1d-91d3fe32e324","task_revision_id":"1","upstream_id":"","short_description":"Early theories of mass communication suggested that audiences were passive…","config":"all","split":"test","body":"{\"choices\":\"['Audiences are more vulnerable and passive than ever before.', 'Television viewing, in particular, creates increased activity in the right hemisphere of the brain.', 'Active receivers are not uniformly affected by mass communication messages.', 'Technology has led to increasingly passive recipients at the ends of message transmission systems.']\",\"question\":\"Early theories of mass communication suggested that audiences were passive recipients of media messages and thus vulnerable to manipulation. Which of the following best describes current thinking?\",\"subject\":\"public_relations\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}