{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"056c28ab-d6fa-504a-bcb7-b33ff62d7618","task_key":"all--test--056c28ab-d6fa-504a-bcb7-b33ff62d7618","task_revision_id":"1","upstream_id":"","short_description":"A brick mason was hired by a builder under a written one-year contract, at an…","config":"all","split":"test","body":"{\"choices\":\"['$20,000 (the $45,000 contract price less the $25,000 the mason could have earned in other employment).', '$45,000 (the contract price).', 'Nothing, because the builder did not act in bad faith when it discharged the mason.', 'Nothing, because the mason did not mitigate his damages.']\",\"question\":\"A brick mason was hired by a builder under a written one-year contract, at an annual salary of $45,000, with employment to begin on March 1. Because the builder was unable to secure enough building contracts to keep all its employees busy during the season beginning March 1, it notified the brick mason on February 15 that it could not afford to employ him as a mason. At the same time, however, the builder offered to employ the mason, for the same contract period, as a night guard at an annual salary of $25,000. The mason declined the offer and remained unemployed during the year. No offer and remained unemployed during the year. No employment for brick masons was available in the community during the year, but the mason could have obtained other employment as a day laborer that would have paid up to $25,000 a year. At the end of the year, in an action against the builder for breach of contract, how much, if anything, is the mason entitled to recover?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}