{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"058d68cb-4108-516e-a987-834fb6fe46d9","task_key":"default--test--1583","task_revision_id":"1","upstream_id":"1583","short_description":"A man conveys to his wife's cousin a property to \"cousin and the heirs of his…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The conveyance gives the niece nothing because the property was already conveyed in fee simple to his wife's cousin.\\\", 'She has an executory interest.', 'She has a fee simple.', 'She has a vested remainder.', 'She gets a leasehold interest.', \\\"She has a partial right, shared with the cousin's heirs.\\\", 'A contingent remainder based upon whether or not cousin has heirs.', 'She gets a life estate.', 'She gets a reversion in case the cousin has no heir of his body at his death.']\",\"question\":\"A man conveys to his wife's cousin a property to \\\"cousin and the heirs of his body.\\\" He then conveys the same property to his niece in fee simple. Under modern real estate laws and practice in most states today, what interest if any does his niece have?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}