{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27bb264d-6375-58ba-b32f-11fe7b2d9d1c","task_key":"all--test--27bb264d-6375-58ba-b32f-11fe7b2d9d1c","task_revision_id":"1","upstream_id":"","short_description":"Students are conducting a plant pigment separation exercise that requires them…","config":"all","split":"test","body":"{\"choices\":\"['Putting on fire-retardant gloves to carry the chamber to a fume hood if the acetone catches fire', 'Placing the chamber in a closed, darkened cabinet for the duration of the pigment separation', 'Wearing leather gloves and covering the nose and mouth with a particle mask', 'Wearing goggles and nitrile-type gloves and keeping all acetone containers closed']\",\"question\":\"Students are conducting a plant pigment separation exercise that requires them to use a closed chromatography chamber with a small amount of acetone in the bottom of the chamber. Safety warnings for acetone include the statement that it is an eye and skin irritant and has flammable vapors. Of the following‚ which safety precaution should the students take while conducting the exercise?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}