{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba02706e-f86b-5bee-921e-ad421724611e","task_key":"default--test--4336","task_revision_id":"1","upstream_id":"4336","short_description":"A 25 g sample of a liquid was heated to 100 °C and then quickly transferred to…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['The final temperature is less than the average starting temperature of the sample and the water; therefore the total energy of the sample and water decreased.', 'The final temperature is greater than the average starting temperature of the sample and the water; therefore the total energy of the sample and water increased.', 'The sample temperature changed more than the water temperature did; therefore the CP of the sample must be greater than the CP of the water.', 'The sample temperature changed less than the water temperature did, but the sample lost the same amount of heat energy as the water gained.', 'The sample temperature changed more than the water temperature did; therefore the sample lost more heat energy than the water gained.', 'The sample temperature changed more than the water temperature did; therefore the sample lost less heat energy than the water gained.', 'The sample temperature changed less than the water temperature did; therefore the CP of the sample must be less than the CP of the water.', 'The sample temperature did not change; therefore the sample did not lose any heat energy.', 'The sample temperature changed more than the water temperature did, but the sample lost the same amount of heat energy as the water gained.', 'The final temperature is the same as the average starting temperature of the sample and the water; therefore the total energy of the sample and water remained the same.']\",\"question\":\"A 25 g sample of a liquid was heated to 100 °C and then quickly transferred to an insulated container holding 100 g of water at 22 °C. The temperature of the mixture rose to reach a final temperature of 35 °C. Which of the following can be concluded?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}