{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9342499a-07d2-5386-8c9c-d7fd350cb10e","task_key":"all--test--9342499a-07d2-5386-8c9c-d7fd350cb10e","task_revision_id":"1","upstream_id":"","short_description":"On February 1, an owner of six vacation cottages in a resort area hired a…","config":"all","split":"test","body":"{\"choices\":\"['The owner would only be obligated to pay for the painting of the four cottages that the caretaker approved.', \\\"The caretaker's approval of the painting of the two cottages destroyed by the wild fire would be excused.\\\", 'The owner would be obligated to pay for the painting of all of the cottages.', \\\"The impossibility of securing the caretaker's approval would render the owner absolutely liable for all of the painting.\\\"]\",\"question\":\"On February 1, an owner of six vacation cottages in a resort area hired a painter to paint all six cottages for $50,000. The contract stipulated that no money would be due until the owner's caretaker approved the work done by the painter. The painter completed the painting of all six cottages. Two days after the painter had finished the painting, a wildfire destroyed the six cottages. Although the caretaker had inspected four of the cottages to his approval, he had not inspected the other two cottages prior to their destruction. Which of the following is the LEAST accurate statement with respect to the caretaker's approval of the painting of the cottages?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}