{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b46beb6e-3d83-584e-b72e-f738f650aefb","task_key":"all--test--b46beb6e-3d83-584e-b72e-f738f650aefb","task_revision_id":"1","upstream_id":"","short_description":"A man owned real estate in fee simple. He transferred a life estate in it over…","config":"all","split":"test","body":"{\"choices\":\"['No, the life estate was broken when the owner of the fee simple interest died.', 'No, the life estate owner is not allowed to convey his interest to other persons.', 'Yes, the life estate owner became the owner in fee simple when the original fee simple owner predeceased him.', 'Yes, the life estate owner could convey his life estate to third persons.']\",\"question\":\"A man owned real estate in fee simple. He transferred a life estate in it over to a friend for the friend's lifetime. When the fee simple owner died, the life estate owner conveyed his interest to his children. The deceased fee simple owner's son sued the children on the basis that the life estate was now broken and he owned the property absolutely. Do the children of the life estate owner have an enforceable interest in the property?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}