{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b75bb4a7-b2ea-5991-ad52-bc4f99e76373","task_key":"default--test--960","task_revision_id":"1","upstream_id":"960","short_description":"A man owned real estate in fee simple. He transferred a life estate in it over…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, the life estate was broken when the owner of the fee simple interest died.', 'No, the life estate owner is not allowed to convey his interest to other persons.', 'Yes, the life estate owner became the owner in fee simple when the original fee simple owner predeceased him.', 'Yes, the children of the life estate owner now own the property in fee simple.', \\\"No, the life estate terminated upon the death of the fee simple owner, and the property reverts back to the fee simple owner's estate.\\\", 'No, the children of the life estate owner have a leasehold interest, not a life estate.', 'Yes, the life estate owner could convey his life estate to third persons.', 'No, the life estate was broken when the life estate owner conveyed his interest to his children.', \\\"Yes, the life estate owner's children inherited the life estate upon their father's death.\\\"]\",\"question\":\"A man owned real estate in fee simple. He transferred a life estate in it over to a friend for the friend's lifetime. When the fee simple owner died, the life estate owner conveyed his interest to his children. The deceased fee simple owner's son sued the children on the basis that the life estate was now broken and he owned the property absolutely. Do the children of the life estate owner have an enforceable interest in the property?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}