{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c86b53e-8585-5f17-9c27-5e5c4f38e8b8","task_key":"all--test--4c86b53e-8585-5f17-9c27-5e5c4f38e8b8","task_revision_id":"1","upstream_id":"","short_description":"Some strains of viruses can change normal mammalian cells into cancer cells in…","config":"all","split":"test","body":"{\"choices\":\"['formation of a pilus between the mammalian cell and the virus', \\\"incorporation of the viral genome into the mammalian cell's nuclear DNA\\\", \\\"conversion of the host's genome into the viral DNA\\\", 'release of spores into the mammalian cell']\",\"question\":\"Some strains of viruses can change normal mammalian cells into cancer cells in vitro. This transformation of the mammalian cell is usually associated with the\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}