{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b33131e9-bd8c-5217-9f5f-741502f814c5","task_key":"all--test--b33131e9-bd8c-5217-9f5f-741502f814c5","task_revision_id":"1","upstream_id":"","short_description":"A graduate from law school had decided against taking the bar exam. Her mother,…","config":"all","split":"test","body":"{\"choices\":\"['an enforceable promise, binding the uncle as a surety.', \\\"an unenforceable promise, because the graduate's mother had a pre-existing duty to pay the graduate.\\\", 'a voidable promise as violative of the statute of frauds.', 'a void promise at the time of inception.']\",\"question\":\"A graduate from law school had decided against taking the bar exam. Her mother, who had always dreamed of having a child who was a lawyer, told the graduate, \\\"It would mean a lot to me if you at least gave the bar exam a try. Tell you what; if you will devote the next two months to studying for the bar exam and then take the exam, I'll give you $2,000. And if you pass I'll give you an additional$2,000. \\\" The graduate replied, \\\"You've got a deal! I'm going to get started studying today!\\\" The graduate's uncle was present when this conversation took place. After the mother had left the room, the uncle said to the graduate, \\\"I want you to know that if anything ever happens to your mother, I'll pay you as per your mother's promise. \\\"The graduate spent the next two months studying diligently for the bar exam. When the graduate returned home after taking the exam, she received the tragic news that her mother had died suddenly. At the mother's funeral, the uncle approached the graduate and told her that he did not believe that he was obligated to pay anything to the graduate. The graduate passed the bar exam and wishes to collect from the uncle. The uncle's promise to the graduate would constitute\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}