{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aab5c10f-6c5b-5980-a67a-87fdfc93bc72","task_key":"default--test--1224","task_revision_id":"1","upstream_id":"1224","short_description":"A college student and her boyfriend lived together in a one-bedroom apartment in…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the co-worker and the student as tenants in common.', 'the boyfriend and the student as joint tenants with right of survivorship.', 'the co-worker and the boyfriend as joint tenants with right of survivorship.', 'the boyfriend as owner in fee simple.', 'the co-worker and the student as joint tenants with right of survivorship.', 'the co-worker and the boyfriend as tenants in common.', 'the co-worker, the student and the boyfriend as joint tenants with right of survivorship.', 'the co-worker as owner in fee simple.', 'the student as owner in fee simple.', 'the co-worker, the student and the boyfriend as tenants in common.']\",\"question\":\"A college student and her boyfriend lived together in a one-bedroom apartment in Los Angeles. They were engaged to be married when they purchased an undeveloped parcel of land on a hillside overlooking the Pacific. The deed, which was properly executed and recorded, named the student and the boyfriend as grantees, \\\"not as tenants in common but as joint tenants with right of survivorship. \\\"Thereafter, the boyfriend, who was experiencing financial difficulties, offered to sell the property to his co-worker. Without the student's knowledge, the boyfriend executed a deed that purported to convey the hillside property to the co-worker in fee simple. The boyfriend signed his name and forged the student's name. He then delivered the deed of conveyance to the co-worker, who paid the boyfriend $150,000, which was the fair market value of the property. The co-worker immediately recorded the deed received from the boyfriend. The common law joint tenancy is unmodified by statute. Title to the property in question is now in \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}