{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd7d3e9b-d214-56ec-a2c8-9b07cf70249a","task_key":"all--test--cd7d3e9b-d214-56ec-a2c8-9b07cf70249a","task_revision_id":"1","upstream_id":"","short_description":"A debtor owed a creditor $15,000 on a debt that had been discharged by the…","config":"all","split":"test","body":"{\"choices\":\"['nothing.', '$10,000. 00', '$10,000, only if the debtor is successful in suing the person who had contracted to buy his house.', '$15,000. 00']\",\"question\":\"A debtor owed a creditor $15,000 on a debt that had been discharged by the debtor's bankruptcy the previous year. The debtor wrote a letter to the creditor stating that he would pay the creditor $10,000 received from the proceeds of the sale of his house in payment of the discharged debt. One week later, the debtor learned that the person who had contracted to buy his house reneged on the deal. As a result, the debtor refused to pay anything to the creditor. If the creditor sues the debtor for breach of contract, he should be entitled to recover\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}