{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35c61b1f-b7ed-5053-a9a0-8db9f1d9fa06","task_key":"all--test--35c61b1f-b7ed-5053-a9a0-8db9f1d9fa06","task_revision_id":"1","upstream_id":"","short_description":"A 25 g sample of a liquid was heated to 100 °C and then quickly transferred to…","config":"all","split":"test","body":"{\"choices\":\"['The sample temperature changed more than the water temperature did; therefore the sample lost more heat energy than the water gained.', 'The sample temperature changed more than the water temperature did, but the sample lost the same amount of heat energy as the water gained.', 'The sample temperature changed more than the water temperature did; therefore the CP of the sample must be greater than the CP of the water.', 'The final temperature is less than the average starting temperature of the sample and the water; therefore the total energy of the sample and water decreased.']\",\"question\":\"A 25 g sample of a liquid was heated to 100 °C and then quickly transferred to an insulated container holding 100 g of water at 22 °C. The temperature of the mixture rose to reach a final temperature of 35 °C. Which of the following can be concluded?\",\"subject\":\"high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}