{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b1974bb-c279-5319-9ee9-566ccafb07bd","task_key":"all--test--4b1974bb-c279-5319-9ee9-566ccafb07bd","task_revision_id":"1","upstream_id":"","short_description":"A 25 g sample of a liquid was heated to 95 °C and then quickly transferred to an…","config":"all","split":"test","body":"{\"choices\":\"['The sample lost more heat energy than the water gained because the sample temperature changed more than the water temperature did.', 'The final temperature is less than the average starting temperatures; therefore the equilibrium constant must be less than 1.', 'The sample temperature changed more than the water temperature did; therefore the heat capacity of the sample must be greater than the heat capacity of the water.', 'Even though the sample temperature changed more than the water temperature did, the sample lost the same amount of heat energy as the water gained to obey the law of conservation of energy.']\",\"question\":\"A 25 g sample of a liquid was heated to 95 °C and then quickly transferred to an insulated container holding 100 g of water at 26 °C. The temperature of the mixture rose to reach a final temperature of 37 °C. Which of the following can be concluded?\",\"subject\":\"high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}