{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1642d2c5-8458-5a2a-9cd5-a77f29107f6f","task_key":"all--test--1642d2c5-8458-5a2a-9cd5-a77f29107f6f","task_revision_id":"1","upstream_id":"","short_description":"Historically, it has been observed that computer processing speeds tend to…","config":"all","split":"test","body":"{\"choices\":\"['Technology companies can accurately predict the dates when new computing innovations will be available to use.', 'Technology companies can plan to double the costs of new products each time advances in processing speed occur.', 'Technology companies can set research and development goals based on anticipated processing speeds.', 'Technology companies can spend less effort developing new processors because processing speed will always improve at the observed rate.']\",\"question\":\"Historically, it has been observed that computer processing speeds tend to double every two years. Which of the following best describes how technology companies can use this observation for planning purposes?\",\"subject\":\"high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}