{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed6aa0c6-407e-5b8f-ac1b-917c84ec87b1","task_key":"all--test--ed6aa0c6-407e-5b8f-ac1b-917c84ec87b1","task_revision_id":"1","upstream_id":"","short_description":"Defendant was arrested on February 1 and released one month later on March 1…","config":"all","split":"test","body":"{\"choices\":\"['discharged because more than 175 days passed between arrest and the filing of the motion to discharge.', 'discharged because more than 175 days passed between his release from jail and the filing of the motion to discharge.', 'brought to trial within 90 days of the filing of the motion to discharge.', 'brought to trial within 10 days of the hearing on the motion to discharge.']\",\"question\":\"Defendant was arrested on February 1 and released one month later on March 1 after being charged with a felony. On December 1 of the same year as his arrest, he filed a motion to discharge since no trial or other action had occurred to that point. The court held a hearing 3 days after the motion was filed. Defendant should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}