{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f6a9103-d566-54a7-b13b-70647e8579fe","task_key":"default--test--1178","task_revision_id":"1","upstream_id":"1178","short_description":"A deed executed by a grantor conveyed his property for a consideration of one…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the brother paid valuable consideration for his deed.', \\\"the grantor's knowledge is imputed to his children.\\\", \\\"the brother is the grantor's sibling.\\\", \\\"the grantor's sons are not protected by the recording statute.\\\", \\\"the grantor's sons did not have a valid claim to the property.\\\", 'the brother had a life interest in the property.', \\\"the brother recorded his deed before the grantor's children sought to oust him from the land.\\\", \\\"the brother's deed was not void due to the recording statute.\\\", \\\"the property's total worth was more than the life interest of the brother.\\\"]\",\"question\":\"A deed executed by a grantor conveyed his property for a consideration of one dollar, receipt of which was acknowledged, \\\"to my brother for life, then to the heirs of my brother. \\\" A life interest in the property for the life of the brother is worth $20,000 on the date of the conveyance. The total worth of the property is $50,000. The brother accepted but didn't record the deed. The recording statute in this jurisdiction provided \\\"unless recorded, all written instruments affecting title to land are void as to subsequent purchasers who paid value and without notice. \\\"Four years later, the grantor purported to convey his property in fee simple absolute to his two sons, by a warranty deed, as a gift. The two sons recorded the deed. Shortly thereafter, the brother ascertained that the grantor's sons were about to take possession of the property. As a consequence, the brother promptly recorded his deed. In a dispute between the brother and the grantor's children as to the ownership of the property, if the brother prevails it will be because\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}