{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eaefce80-4626-5883-b83c-0b7755d2952a","task_key":"all--test--eaefce80-4626-5883-b83c-0b7755d2952a","task_revision_id":"1","upstream_id":"","short_description":"A rigid, solid container of constant volume holds an ideal gas of volume v1 and…","config":"all","split":"test","body":"{\"choices\":\"['The average speed of the molecules increases.', 'The pressure increases.', 'The kinetic energy of the system increases.', 'The volume increases.']\",\"question\":\"A rigid, solid container of constant volume holds an ideal gas of volume v1 and temperature T1 and pressure P1. The temperature is increased in an isochoric process. Which of the following is NOT true?\",\"subject\":\"high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}