{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14742030-6cef-5a91-b8e1-688c317dc24f","task_key":"default--test--1844","task_revision_id":"1","upstream_id":"1844","short_description":"A professor, being fee simple owner of an apartment building, made a conveyance…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"the son's estate, subject to the professor's life estate.\\\", \\\"the daughter and the doctor as tenants in common, subject to the professor's life estate.\\\", \\\"the daughter as the sole owner, subject to the professor's life estate.\\\", 'the daughter and the doctor as tenants in common.', 'the professor as the sole owner.', \\\"the daughter subject to the professor's life estate.\\\", \\\"the doctor as the sole owner, subject to the professor's life estate.\\\", \\\"the daughter and the professor as joint tenants with survivorship rights, subject to the professor's life estate.\\\", \\\"the doctor and the professor as joint tenants with survivorship rights, subject to the professor's life estate.\\\"]\",\"question\":\"A professor, being fee simple owner of an apartment building, made a conveyance thereof in these words, \\\"I hereby convey my apartment building to my son and daughter as tenants in common. \\\" In the deed instrument, which was properly executed, the professor reserved a life estate to himself. The professor then told his children of his conveyance and put it in the family vault in the library for safekeeping. Thereafter, the son married a doctor. The professor, who disliked the doctor, then executed another deed that he labeled \\\"a correction deed. \\\" In the \\\"correction deed,\\\" the professor conveyed the apartment building \\\"to my son and daughter as joint tenants with right of survivorship. \\\" According to the new deed, the professor again reserved a life estate to himself. Both children accepted delivery of the \\\"correction deed. \\\" Six months later, the son died intestate, leaving the doctor as his sole heir. Title to the apartment building is in\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}