{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"783d0b2e-1131-5e67-ba28-f21acf5e4a9f","task_key":"all--test--783d0b2e-1131-5e67-ba28-f21acf5e4a9f","task_revision_id":"1","upstream_id":"","short_description":"A woman owned an extensive art collection that she displayed in a special room…","config":"all","split":"test","body":"{\"choices\":\"['The woman, because her promise was gratuitous.', 'The woman, because the insurance company owed her a pre-existing duty to find the painting.', \\\"The detective, because he did the act necessary to constitute an acceptance of the woman's offer.\\\", \\\"The detective, because the market value of the painting exceeded its insured value, so there was sufficient consideration to support the woman's promise.\\\"]\",\"question\":\"A woman owned an extensive art collection that she displayed in a special room of her home. While the woman was away on a vacation, there was a burglary at her home, and her favorite painting was stolen. Although the painting was insured for $1,000,000 by an insurance company, it had a market value of over $1,500,000. When the woman returned from vacation, she met with a detective employed by the insurance company to investigate the theft. During their meeting, the woman told the detective that she would pay him an extra $50,000 if he recovered the paihting. For the next three weeks, the detective investigated the theft as part of his job responsibilities with the insurance company. Within the course of this investigation, the detective learned who was responsible for the burglary. As a consequence, the culprit was apprehended, and the painting was recovered and returned to the woman. The detective then requested the $50,000 that the woman had promised to pay him. After the woman refused to make the payment, the detective sued the woman for breach of contract. Who is most likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}