{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60c3a605-ebdc-574e-8401-a413fb8e6b4d","task_key":"all--test--60c3a605-ebdc-574e-8401-a413fb8e6b4d","task_revision_id":"1","upstream_id":"","short_description":"A furniture dealer sold furniture to a young couple with less than perfect…","config":"all","split":"test","body":"{\"choices\":\"['No, because they might be able to find another store to sell them furniture, which proves that there was no lack of bargaining power.', 'No, because the store was nice enough to extend credit, and the couple should be expected to pay for everything before they own any of it.', 'Yes, because any time a seller puts terms in fine print it is proof of bad faith and unconscionability.', 'Yes, because a combination of factors makes it likely that the court will recognize unconscionability under these circumstances.']\",\"question\":\"A furniture dealer sold furniture to a young couple with less than perfect credit. They signed a contract that said that if they purchased new items on the account, they would not own the old purchases until the new ones were paid in full. That provision was in fine print on the reverse side of the papers. When husband lost his job, they had by that time paid for everything purchased on the account except for one chair they bought a few weeks earlier. The store sued, trying to repossess all furniture ever sold to the couple. Will the couple likely prevail on a defense of unconscionability?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}