{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f4c27b9-8d99-528c-9cd6-afc4bf5d197e","task_key":"all--test--8f4c27b9-8d99-528c-9cd6-afc4bf5d197e","task_revision_id":"1","upstream_id":"","short_description":"A grantor executed a deed by which he conveyed his apartment building for a…","config":"all","split":"test","body":"{\"choices\":\"['estate pur autre vie.', 'contingent remainder pur autre vie.', 'vested remainder pur autre vie.', 'shifting executory interest pur autre vie.']\",\"question\":\"A grantor executed a deed by which he conveyed his apartment building for a consideration of one dollar, receipt of which was acknowledged, \\\"to my son for life, then to my uncle for life, but if my uncle moves to another state, to my brother for the life of my uncle, then to the heirs of my uncle if my uncle does not move to another state, and to the heirs of my brother if my uncle does move to another state. \\\" This deed was promptly recorded. During the son's lifetime, the brother's interest may best be described as a (an)\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}