{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b2fd4f0-3a80-5d9f-b8b0-9b733641919c","task_key":"all--test--9b2fd4f0-3a80-5d9f-b8b0-9b733641919c","task_revision_id":"1","upstream_id":"","short_description":"Which of the following must be true about a binary liquid mixture that obeys…","config":"all","split":"test","body":"{\"choices\":\"['I only', 'III only', 'I and III only', 'I, II, and III']\",\"question\":\"Which of the following must be true about a binary liquid mixture that obeys Raoult’s law?\\nI. The partial pressure of each component at equilibrium is proportional to its mole fraction in the liquid mixture.\\nII. The volume of the mixture is equal to the sum of the volumes of each component before mixing.\\nIII. Intermolecular interactions in the mixture are identical to intermolecular interactions in the pure components.\",\"subject\":\"college_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}