{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6aa76bb8-d94c-534c-bc2c-8cb902df5a75","task_key":"all--test--6aa76bb8-d94c-534c-bc2c-8cb902df5a75","task_revision_id":"1","upstream_id":"","short_description":"A certain computer game is played between a human player and a…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The game's running speed can only be improved if the game is played between two human players instead of with the computer-controlled player.\\\", \\\"The game's running speed might be improved by using a process that finds approximate solutions every time the computer-controlled player has a turn.\\\", \\\"The game's running speed cannot be improved because computers can only be programmed to find the best possible solution.\\\", \\\"The game's running speed cannot be improved because the game is an example of an algorithm that does not run in a reasonable time.\\\"]\",\"question\":\"A certain computer game is played between a human player and a computer-controlled player. Every time the computer-controlled player has a turn, the game runs slowly because the computer evaluates all potential moves and selects the best one. Which of the following best describes the possibility of improving the running speed of the game?\",\"subject\":\"high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}