{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e931b911-1b7a-530a-bb4a-a01cd1081ebb","task_key":"all--test--e931b911-1b7a-530a-bb4a-a01cd1081ebb","task_revision_id":"1","upstream_id":"","short_description":"An engineer signed a two-year contract to serve as the chief safety engineer for…","config":"all","split":"test","body":"{\"choices\":\"['No, because an at-will employee has the right to terminate an employment contract.', \\\"No, because the risk to the engineer's health excused his nonperformance of the contract.\\\", \\\"Yes, because the coal mine acted in good faith in mitigating the effect of the engineer's failure to finish the contract term.\\\", 'Yes, because the mine is reasonably safe for most people.']\",\"question\":\"An engineer signed a two-year contract to serve as the chief safety engineer for a coal mine at a salary of $7,000 per month. The position required the engineer to work underground each workday. After one week on the job, the engineer became very ill. He requested testing of the mine air system, which revealed the presence of a chemical agent to which the engineer had a rare allergic reaction. The engineer promptly quit. The coal mine then hired a qualified replacement for the remainder of the engineer's contract at a salary of $7,500 per month. Assume that no statute or regulation applies. If the coal mine sues the engineer for breach of contract, is the coal mine likely to recover damages?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}