{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22356a45-5f93-5d1c-90ee-761e598b6d3f","task_key":"all--test--22356a45-5f93-5d1c-90ee-761e598b6d3f","task_revision_id":"1","upstream_id":"","short_description":"A rancher, being owner in fee simpleconveyed the property by warranty deed to a…","config":"all","split":"test","body":"{\"choices\":\"[\\\"succeed, because an implied delegation of duties resulted from the woman's conveyance to the farmer.\\\", 'succeed, because the niece was a third-party beneficiary under the conveyance between the woman and the farmer.', 'not succeed, because the farmer did not promise to pay the mortgage debt.', 'not succeed, unless the farmer had constructive notice of the existence of the mortgage.']\",\"question\":\"A rancher, being owner in fee simpleconveyed the property by warranty deed to a woman. The woman gave her niece a mortgage on the ranch to secure a loan from the niece to the woman in the amount of $500,000. The mortgage was recorded immediately. Two years later, the woman conveyed the ranch to a farmer by quitclaim deed. The woman then defaulted on the mortgage, and the niece brought an in personam action against the farmer to recover the amount of the mortgage due. Assume that the woman's quitclaim deed to the farmer made no reference to the mortgage. The woman then defaulted on the mortgage, and the niece brought an in personam action against the farmer to recover the amount of the mortgage due. The mortgagee will probably\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}