{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb1f882f-9e71-5837-8359-00fe1208a95f","task_key":"all--test--cb1f882f-9e71-5837-8359-00fe1208a95f","task_revision_id":"1","upstream_id":"","short_description":"A woman attended a private university. One afternoon she was approached by a…","config":"all","split":"test","body":"{\"choices\":\"['hear the case.', 'dismiss the action as moot.', 'dismiss the action since the issues are no longer ripe.', 'dismiss the action because there is no case or controversy.']\",\"question\":\"A woman attended a private university. One afternoon she was approached by a photographer for a magazine, who asked her if she would be interested in posing nude for an upcoming issue. The woman, who was an aspiring model, agreed and posed for a number of nude pictures, which subsequently appeared in the magazine. Afterward, administrators and professors at the university began to harass her for what they considered to be imprudent behavior. During class, her instructors frequently called her \\\"Playmate\\\" and \\\"Stripper. \\\" Consequently, the woman brought suit in federal court against the university, alleging harassment, and seeking an injunction and damages. After this action was instituted, the university signed a stipulation agreeing not to harass her in the future. The federal court should now\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}