{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"717259e1-5a6e-5d9f-8dc5-e39b0c17e6aa","task_key":"all--test--717259e1-5a6e-5d9f-8dc5-e39b0c17e6aa","task_revision_id":"1","upstream_id":"","short_description":"A circuit consists of a battery and a light bulb. At first, the circuit is…","config":"all","split":"test","body":"{\"choices\":\"['The net charge has become more positive.', 'The net charge has become more negative.', 'The net charge has not changed.', 'Whether the net charge becomes more positive or more negative depends on the initial net charge residing on the circuit elements before the bulb was lit.']\",\"question\":\"A circuit consists of a battery and a light bulb. At first, the circuit is disconnected. Then, the circuit is connected, and the light bulb lights. After the light bulb has been lit for a few moments, how has the net charge residing on the circuit elements changed?\",\"subject\":\"high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}