{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e587369e-b755-56c3-9da8-d0f15a9681a8","task_key":"default--test--1841","task_revision_id":"1","upstream_id":"1841","short_description":"A brother and a sister purchased land under a deed that conveyed title to them…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Title is entirely in the cousin as the beneficiary of the brother's will.\\\", 'The answer depends on whether the jurisdiction follows the lien theory or the title theory ofmortgages.', \\\"The brother's share of the land is in probate, while the sister retains her original share.\\\", 'Title is in the sister, the cousin, and the mortgage lender as tenants in common.', 'Title is in the sister and the cousin as tenants in common, with the sister having a 40% interest and the cousin having a 60% interest.', 'Title is in the sister and the cousin as equal tenants in common.', 'Title is in the sister and the cousin as joint tenants with right of survivorship.', 'Title is in the sister as she has paid off her mortgage debt.', 'Title is in the sister and the cousin as tenants in common, with the sister having a 20% interest and thecousin having an 80% interest.']\",\"question\":\"A brother and a sister purchased land under a deed that conveyed title to them as joint tenants with right ofsurvivorship. Common law joint tenancy is unmodified by statute in the jurisdiction. The purchase price was $50,000, of which the sister paid $10,000 and the brother paid $40,000. The sisterlater mortgaged her interest in the land. The brother then died testate, leaving his entire estate to a cousin. Thesister later paid off her mortgage debt, and the mortgage was released. At the present time, who owns the land?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}