{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0e97746-95be-55e9-981b-a3fea12d3a28","task_key":"all--test--a0e97746-95be-55e9-981b-a3fea12d3a28","task_revision_id":"1","upstream_id":"","short_description":"The relationship between ethics and the law can be described as…","config":"all","split":"test","body":"{\"choices\":\"['Incredibly close, in fact, in all instances what is ethical is also what is legal and vice versa.', 'Not close, ethics and the law share a small amount of overlap, but fundamentally address differing questions and themes.', 'Close, as there is considerable overlap between ethics and law, but the two are not equivalent.', 'Not at all close, ethics and the law are virtually incompatible.']\",\"question\":\"The relationship between ethics and the law can be described as…\",\"subject\":\"business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}