{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ec249c3-d329-5ba3-bb03-ba0ef51e2a96","task_key":"all--test--4ec249c3-d329-5ba3-bb03-ba0ef51e2a96","task_revision_id":"1","upstream_id":"","short_description":"A brother and a sister purchased land under a deed that conveyed title to them…","config":"all","split":"test","body":"{\"choices\":\"['The answer depends on whether the jurisdiction follows the lien theory or the title theory ofmortgages.', 'Title is entirely in the sister as the surviving joint tenant.', 'Title is in the sister and the cousin as equal tenants in common.', 'Title is in the sister and the cousin as tenants in common, with the sister having a 20% interest and thecousin having an 80% interest.']\",\"question\":\"A brother and a sister purchased land under a deed that conveyed title to them as joint tenants with right ofsurvivorship. Common law joint tenancy is unmodified by statute in the jurisdiction. The purchase price was $50,000, of which the sister paid $10,000 and the brother paid $40,000. The sisterlater mortgaged her interest in the land. The brother then died testate, leaving his entire estate to a cousin. Thesister later paid off her mortgage debt, and the mortgage was released. At the present time, who owns the land?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}