{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5b37fa1-ab83-589b-ad59-225c5813524f","task_key":"all--test--e5b37fa1-ab83-589b-ad59-225c5813524f","task_revision_id":"1","upstream_id":"","short_description":"A user enters a Web address in a browser, and a request for a file is sent to a…","config":"all","split":"test","body":"{\"choices\":\"['The file is broken into packets for transmission. The packets must be reassembled upon receipt.', \\\"The file is broken into packets for transmission. The user's browser must request each packet in order until all packets are received.\\\", \\\"The server attempts to connect directly to the user's computer. If the connection is successful, the entire file is sent. If the connection is unsuccessful, an error message is sent to the user.\\\", \\\"The server repeatedly attempts to connect directly to the user's computer until a connection is made. Once the connection is made, the entire file is sent.\\\"]\",\"question\":\"A user enters a Web address in a browser, and a request for a file is sent to a Web server. Which of the following best describes how the file is sent to the user?\",\"subject\":\"high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}