{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a47c87e-f779-5772-af1e-57b5e0b948e1","task_key":"all--test--8a47c87e-f779-5772-af1e-57b5e0b948e1","task_revision_id":"1","upstream_id":"","short_description":"A deed executed by a woman in 2001 conveyed a tract of land for a consideration…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The friend's heirs.\\\", 'The cousin.', 'The woman.', \\\"Divided equally between the friend's heirs and the woman.\\\"]\",\"question\":\"A deed executed by a woman in 2001 conveyed a tract of land for a consideration of one dollar, receipt of which was acknowledged, \\\"to my friend for life, but if liquor is ever sold on the tract of land, then to my cousin and his heirs, and if for any reason the interest hereby conveyed to my cousin is not valid, then I reserve the right to re-enter the tract of land and take back my property. \\\" In 2004, the friend died intestate before the wheat he had planted could be harvested. Who is entitled to the proceeds of the crop?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}