{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c959b39-2693-5070-8c2a-1236eb7a49ab","task_key":"all--test--5c959b39-2693-5070-8c2a-1236eb7a49ab","task_revision_id":"1","upstream_id":"","short_description":"A student is recording a song on her computer. When the recording is finished,…","config":"all","split":"test","body":"{\"choices\":\"['The song was saved using fewer bits per second than the original song.', 'The song was saved using more bits per second than the original song.', 'The song was saved using a lossless compression technique.', 'Some information is lost every time a file is saved from one location on a computer to another location.']\",\"question\":\"A student is recording a song on her computer. When the recording is finished, she saves a copy on her computer. The student notices that the saved copy is of lower sound quality than the original recording. Which of the following could be a possible explanation for the difference in sound quality?\",\"subject\":\"high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}