{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68608e31-68db-5fee-90ee-b4de2a33fba4","task_key":"default--test--1035","task_revision_id":"1","upstream_id":"1035","short_description":"On March 1, a homeowner hired a landscaper to landscape his front yard. On March…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The landscaper's promise created an enforceable bilateral contract.\\\", \\\"The landscaper's promise constituted a breach of the original contract.\\\", \\\"The landscaper's promise created an enforceable unilateral contract.\\\", \\\"The landscaper's promise was enforceable only if the homeowner made a down payment.\\\", \\\"The landscaper's promise was enforceable only if the work was completed by the agreed deadline.\\\", \\\"The landscaper's promise was void, as it was made under duress.\\\", \\\"The landscaper's promise was illusory, thereby creating an unenforceable contract.\\\", \\\"The landscaper's promise resulted in a quasi-contract.\\\", \\\"The landscaper's promise was voidable, since it was not in writing.\\\", \\\"The landscaper's promise was unenforceable due to lack of consideration.\\\"]\",\"question\":\"On March 1, a homeowner hired a landscaper to landscape his front yard. On March 25, as the landscaper was almost finished with the work on the front yard, the homeowner said to the landscaper, \\\"I'm very pleased with your work to date. If you are willing, I would like you to landscape my backyard, on the same terms and conditions as our existing contract, the work to be completed by May 1. If you can meet that deadline, I'll pay you $10,000 at that time. \\\" The landscaper replied, \\\"Agreed. Let's shake hands on it. \\\" They shook hands. What is the probable legal effect of the landscaper's promise on March 25 to landscape the backyard?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}