{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69a205a3-e0d6-53b5-9084-37d35b6a6020","task_key":"all--test--69a205a3-e0d6-53b5-9084-37d35b6a6020","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is the most likely data privacy concern of the upgraded…","config":"all","split":"test","body":"{\"choices\":\"['Customers’ personal information could be compromised if an unauthorized individual gains access to the call session database.', 'Storing information in the call session database makes it easy for individuals to trick the system using malicious links.', 'The system design increases the chance that customers will unknowingly install malware on their devices that will share their data with unauthorized individuals.', 'The system design makes it easy for unauthorized individuals to acquire customers’ private encryption keys.']\",\"question\":\"Which of the following is the most likely data privacy concern of the upgraded system?\",\"subject\":\"high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}