{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e900e55-0ee1-51b6-b5ed-a9d4f879b1a7","task_key":"all--test--9e900e55-0ee1-51b6-b5ed-a9d4f879b1a7","task_revision_id":"1","upstream_id":"","short_description":"The Evergreen State University charges resident students $1,200 per year and…","config":"all","split":"test","body":"{\"choices\":\"['Dismiss the suit as moot.', 'Dismiss the suit, because Imma lacked standing to sue initially.', 'Hear the case if it appears that Ms. Grant is diligently prosecuting the matter.', 'Hear the case due to the amicus interests.']\",\"question\":\"The Evergreen State University charges resident students $1,200 per year and nonresident students $4,800 per year in tuition. The University considered a student a \\\"nonresident\\\" if the student's primary residence was outside the state at any time during the previous school year. Imma Grant transferred from Sunshine State University to Evergreen State in her junior year, and paid nonresident tuition. She filed a declaratory judgment action against the Evergreen Registrar to have her considered a resident and asked for no money damages. Fourteen groups filed amicus briefs in support of Imma, advocating a change in Evergreen's protectionist regulation. Now that she is a senior, and spent the previous year within Evergreen so she is thus now considered a resident student. The federal court should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}