{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f70bcdcd-c04d-5184-83ad-467437c88a14","task_key":"all--test--f70bcdcd-c04d-5184-83ad-467437c88a14","task_revision_id":"1","upstream_id":"","short_description":"A fortune teller told fortunes by means of Tarot cards. An elderly woman, who…","config":"all","split":"test","body":"{\"choices\":\"[\\\"prevail, because she relied to her detriment on the fortune teller's foretelling.\\\", 'prevail, if the fortuneteller did not honestly believe that the woman would live to be 100 years of age.', 'not prevail, unless there was a fiduciary relationship between the parties.', 'not prevail, unless the fortuneteller warranted the truth as believed.']\",\"question\":\"A fortune teller told fortunes by means of Tarot cards. An elderly woman, who was worried about her failing health, had heard that the fortuneteller was clairvoyant and could see into the future. Consequently, the woman decided to see the fortuneteller in order to have her Tarot cards read. As the fortuneteller was telling the woman her fortune, she suddenly said, \\\"I have a vision. If you give me $25,000 tomorrow, you will live to be 100 years old. \\\" The woman, who was 72 years of age, believed the fortuneteller and gave her the money the next day. The following week, the woman's physician informed her that she had a serious heart disease and he didn't expect her to live for more than a year or two. If the woman asserts a claim against the fortuneteller based on deceit, the plaintiff should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}