{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d17ce34-72d9-51e9-acd7-fd2da734c44a","task_key":"all--test--3d17ce34-72d9-51e9-acd7-fd2da734c44a","task_revision_id":"1","upstream_id":"","short_description":"A man conveys to his wife's cousin a property to \"cousin and the heirs of his…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The conveyance gives the niece nothing because the property was already conveyed in fee simple to his wife's cousin.\\\", 'She gets a reversion in case the cousin has no heir of his body at his death.', 'A contingent remainder based upon whether or not cousin has heirs.', 'She has a fee simple.']\",\"question\":\"A man conveys to his wife's cousin a property to \\\"cousin and the heirs of his body.\\\" He then conveys the same property to his niece in fee simple. Under modern real estate laws and practice in most states today, what interest if any does his niece have?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}