{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e885b311-caab-5dd8-8e95-9977dec4cc3a","task_key":"all--test--e885b311-caab-5dd8-8e95-9977dec4cc3a","task_revision_id":"1","upstream_id":"","short_description":"Two people who have been cohabitating decide to get married. The man asks the…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the consideration for the contract was the marriage and the consummation of the marriage made the contract complete and enforceable.', 'Yes, because prenuptial agreements are given the benefit of the doubt when it comes to enforcement decisions.', 'No, because a contract in consideration of marriage must be in writing and the prenuptial was therefore not legal as long as it remained oral.', 'No, because husband did not live up to his promise to put the agreement in writing and therefore he was guilty of fraud.']\",\"question\":\"Two people who have been cohabitating decide to get married. The man asks the woman to agree that all of his property that he now owns will remain in his name if they ever get divorced. She agrees to that but demands a written document. However, they got married without having signed anything. Three months later, the husband leaves the wife for another woman. He sends her notice to vacate the home due to their prenuptial agreement. When she doesn't move, husband sues, asking for enforcement of the oral prenuptial contract. Will the court likely enforce the husband's claim?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}