{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"706cdca4-fb68-575e-a4c1-ad31f624db13","task_key":"default--test--1836","task_revision_id":"1","upstream_id":"1836","short_description":"A homeowner entered into a written contract with a house painter to have his…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The house painter's performance under the contract would be an express condition subsequent to the homeowner's duty of payment of the $10,000.\\\", 'The performances of the homeowner and the house painter were, in essence, concurrent conditions.', 'The performances of the homeowner and the house painter were, in essence, independent conditions.', \\\"Payment of the $10,000 by the homeowner would be a concurrent condition to the house painter's duty of performance.\\\", \\\"The house painter's performance under the contract would be an express condition precedent to the homeowner's duty of payment of the $10,000.\\\", \\\"Payment of the $10,000 by the homeowner would be an express condition subsequent to the house painter's duty of performance.\\\", \\\"Payment of the $10,000 by the homeowner would be an express condition precedent to the house painter's duty of performance.\\\", \\\"The homeowner's satisfaction with the job is an implied condition precedent to the house painter's duty of performance.\\\", \\\"The homeowner's satisfaction with the job would be an express condition precedent to the house painter's duty of performance.\\\", \\\"The homeowner's satisfaction with the job is an implied condition subsequent to the house painter's duty of performance.\\\"]\",\"question\":\"A homeowner entered into a written contract with a house painter to have his house painted for $10,000. The contract stated in part: \\\"It is agreed that the house painter will paint the house using off-white latex paint, and will finish the job within 30 days from the execution of this agreement. If the homeowner finds the completed job to be satisfactory, he will pay the house painter $10,000 within one week after the job is completed. \\\"Which of the following is the most accurate statement regarding the homeowner's contractual obligation to pay the house painter $10,000?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}