{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9624e0c-8125-5353-87bf-fc85a9ea9303","task_key":"all--test--b9624e0c-8125-5353-87bf-fc85a9ea9303","task_revision_id":"1","upstream_id":"","short_description":"A 25 g sample of a solid was heated to 100 °C and then quickly transferred to an…","config":"all","split":"test","body":"{\"choices\":\"['The sample lost more thermal energy than the water gained because the sample temperature changed more than the water temperature did.', 'Even though the sample temperature changed more than the water temperature did, the sample lost the same amount of thermal energy as the water gained.', 'The sample temperature changed more than the water temperature did; therefore the heat capacity of the sample must be greater than the heat capacity of the water.', 'The final temperature is less than the average starting temperatures; therefore the equilibrium constant must be less than 1.']\",\"question\":\"A 25 g sample of a solid was heated to 100 °C and then quickly transferred to an insulated container holding 100 g of water at 26 °C. The temperature of the mixture rose to reach a final temperature of 37 °C. Which of the following can be concluded?\",\"subject\":\"high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}