{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8135330e-db7e-5ad9-b31c-6535b4f38bf3","task_key":"all--test--8135330e-db7e-5ad9-b31c-6535b4f38bf3","task_revision_id":"1","upstream_id":"","short_description":"A landowner, being fee simple owner of an apartment building, devised it \"to…","config":"all","split":"test","body":"{\"choices\":\"['to the estate of the assistant and its successors and assigns.', 'to the heirs and assigns of the assistant.', 'to the daughter and her heirs and assigns.', 'to the assistant as a former tenant in common with remainder to the daughter.']\",\"question\":\"A landowner, being fee simple owner of an apartment building, devised it \\\"to both my assistant and my friend as tenants in common. \\\" Thereafter, the assistant died intestate, leaving his daughter as his only surviving heir. While the assistant's estate was in administration, the friend agreed to convey his interest in the apartment building to the daughter. A valid conveyance by the friend would be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}