{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3d8db11-f004-5b8b-843f-388e38bd26ab","task_key":"all--test--b3d8db11-f004-5b8b-843f-388e38bd26ab","task_revision_id":"1","upstream_id":"","short_description":"A man entered into a written contract with a nationally famous artist. Their…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The man's dissatisfaction with the quality of the portrait was not genuine.\\\", 'The man failed to make an adequate examination of the quality of the portrait.', \\\"The man's dissatisfaction with the portrait was the result of delivery after November 1, rather than of any lack of quality in the portrait.\\\", 'Because of a recent hand injury, the artist did not have sufficient time to produce a portrait of a quality satisfactory to the man.']\",\"question\":\"A man entered into a written contract with a nationally famous artist. Their contract provided that the artist would paint a full-color portrait of the man, to be delivered by November 1, in exchange for $15,000. A contractual provision recited that the artist \\\"guarantees that the quality of the portrait will be fully satisfactory. \\\"The artist finished the portrait and presented it to the man on November 2. The man looked at the portrait and said, \\\"This is awful! I really expected better. I'm not paying for this!\\\" The artist was enraged by the man's reaction, as he felt the finished portrait was one of his best works. Which of the following arguments would be LEAST helpful to the artist in his efforts to recover on a breach of contract theory?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}