{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"454b952f-8315-522f-934d-ca5b6ef3bb30","task_key":"all--test--454b952f-8315-522f-934d-ca5b6ef3bb30","task_revision_id":"1","upstream_id":"","short_description":"Sulfurous acid is a weak acid, while sulfuric acid is a much stronger acid…","config":"all","split":"test","body":"{\"choices\":\"['the sulfur in sulfuric acid is more electronegative than the sulfur in sulfurous acid', 'sulfuric acid has more oxygen atoms in its formula', 'the O–H bonds in sulfuric acid are much weaker than those in sulfurous acid', 'sulfurous acid has its hydrogen atoms bound directly to the sulfur atom']\",\"question\":\"Sulfurous acid is a weak acid, while sulfuric acid is a much stronger acid because\",\"subject\":\"high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}