{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7b845df-cabc-5c00-ac2d-5a84819666b5","task_key":"all--test--f7b845df-cabc-5c00-ac2d-5a84819666b5","task_revision_id":"1","upstream_id":"","short_description":"A law student was accepted by a law school for the fall term. Several…","config":"all","split":"test","body":"{\"choices\":\"['the promise constituted an unenforceable, conditional gift.', 'the promise would not be legally binding, since it was non-detrimental.', 'the promise would be enforceable if a bargained-for exchange was so intended.', 'the promise constituted a voidable proposal.']\",\"question\":\"A law student was accepted by a law school for the fall term. Several generations of the law student's family had attended this prestigious law school. One week before the start of the fall term, at a party to celebrate the law student's acceptance to the school, his father announced to the law student, in the presence of the partygoers, \\\"Son, it's your obligation to uphold the family tradition for excellence at the school. In this regard, if you promise to study a minimum of five hours a day, I shall pay you $1,000 for each €˜A' you achieve during your first year; $2,500 for each €˜A' you achieve during your second year; and $5,000 for each €˜A' you achieve during your third year. \\\"The most accurate statement concerning the father's promise to reward the law student for achieving \\\"A's\\\" would be that\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}