{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a89a7fb3-5937-5662-b398-c1f67b35ed38","task_key":"default--test--1455","task_revision_id":"1","upstream_id":"1455","short_description":"A man borrowed $150,000 from a bank to remodel his home and executed a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, because the bank's loan modification was not detrimental to the rights of the mother.\\\", \\\"Yes, because the man failed to inform his mother about the bank's loan modification.\\\", \\\"Yes, because the bank's loan modification was made without the mother's prior consent.\\\", \\\"No, because the mother did not take action when she learned about the bank's loan modification.\\\", \\\"No, because the man's obligation to repay his mother was not affected by the bank's loan modification.\\\", \\\"No, because the man was not in default in his payments to his mother at the time of the bank's loan modification.\\\", \\\"Yes, because the bank's loan modification was material.\\\", \\\"Yes, because the bank's loan modification extended the repayment period.\\\", 'Yes, because the man was struggling to make his payments to his mother.']\",\"question\":\"A man borrowed $150,000 from a bank to remodel his home and executed a promissory note agreeing to repay the loan over a 10-year period. The loan was secured by a mortgage on the home. The bank promptly recorded the mortgage, which was the only lien on the home at that time. Several months later, the man borrowed $40,000 from his mother in order to purchase a new truck and gave his mother a mortgage on the home to secure repayment of the debt. The mother promptly recorded the mortgage. The man later lost his job and was struggling to make loan payments to both the bank and his mother. To accommodate the man's financial situation, the bank extended the amortization period of its loan to substantially reduce the amount of each monthly payment. The bank did not alter the interest rate or increase the principal amount of the loan. At the time of the modification of the bank loan, the man was not in default in his payments to his mother. Neither the bank nor the man informed the mother of the modification of the bank loan. After the man later missed five payments to his mother, she commenced a foreclosure action. While the action was pending, the mother learned of the bank's loan modification. The mother asserted that her mortgage had become a first lien on the home and that the bank's mortgage had become a second lien. Is the mother's assertion correct?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}