{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab61237f-a41c-5903-880e-4b5a7db69140","task_key":"all--test--ab61237f-a41c-5903-880e-4b5a7db69140","task_revision_id":"1","upstream_id":"","short_description":"Starting with a stock solution of 18.0 M H2SO4, what is the proper procedure to…","config":"all","split":"test","body":"{\"choices\":\"['Add 167 mL of the stock solution to the flask, then fill the flask the rest of the way with distilled water while swirling the solution.', 'Add 600 mL of the stock solution to the flask, then fill the flask the rest of the way with distilled water while swirling the solution.', 'Fill the flask partway with water, then add 167 mL of the stock solution, swirling to mix it. Last, fill the flask the rest of the way with distilled water.', 'Fill the flask partway with water, then add 600 mL of the stock solution, swirling to mix it. Last, fill the flask the rest of the way with distilled water.']\",\"question\":\"Starting with a stock solution of 18.0 M H2SO4, what is the proper procedure to create a 1.00 L sample of a 3.0 M solution of H2SO4 in a volumetric flask?\",\"subject\":\"high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}