{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7ddd416-3ff2-5fa1-99e2-c1b88f852b5f","task_key":"default--test--1330","task_revision_id":"1","upstream_id":"1330","short_description":"A debtor owed a creditor $12,000 under a promissory note. Under the applicable…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['On July 1, not June 1, the creditor will be entitled to a judgment against the debtor for$12,000.', 'On June 1, the creditor became entitled to a judgment against the debtor for $12,000.', 'The creditor is entitled to a judgment against the debtor for $5,000, regardless of the date.', 'On August 1, the creditor became entitled to a judgment against the debtor for $12,000.', 'The creditor is entitled to a judgment against the debtor for $7,000, the remaining amount after the promised $5,000 payment.', \\\"On July 1, the creditor became entitled to a judgment against the debtor for the full $12,000, due to the debtor's failure to pay.\\\", 'The creditor is not entitled to anything, due to the statute of limitations expiring.', 'On July 1, not June 1, the creditor became entitled to a judgment against the debtor for $5,000 only.', 'The creditor is not entitled to anything, on either June 1 or on July 1.']\",\"question\":\"A debtor owed a creditor $12,000 under a promissory note. Under the applicable statute of limitations, a suit to collect on the promissory note had to have been filed by September 30 of last year. On June 1 of this year, the creditor received a letter from the debtor stating, \\\"I shall pay you $5,000 on July 1 in full satisfaction of what I owe you. \\\" However, the debtor failed to make the payment on July 1. If, on August 1, the creditor brings suit against the debtor and the debtor asserts the statute of limitations as an affirmative defense and refuses to pay the creditor anything, which of the following accurately states the creditor's legal rights against the debtor?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}