{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6fb857dd-fb3f-5c55-b275-1cf14ae05f2b","task_key":"all--test--6fb857dd-fb3f-5c55-b275-1cf14ae05f2b","task_revision_id":"1","upstream_id":"","short_description":"A 12-year-old boy decided to play a practical joke. Knowing that his parents…","config":"all","split":"test","body":"{\"choices\":\"[\\\"the boy wins, because a child's conduct cannot be negligent.\\\", 'the boy wins, because the guest unreasonably assumed the risk of entering an unfamiliar room rather than asking for directions to the bathroom.', 'the boy loses, because this \\\"practical joke\\\" posed a risk of severe harm and, therefore, the boy had a duty to exercise as much care as a reasonable adult.', 'the boy loses, because he did not act with the amount of care that one would expect from a 12-year-old child.']\",\"question\":\"A 12-year-old boy decided to play a practical joke. Knowing that his parents were giving a dinner party that evening, the boy filled a bucket with ice-cold water and balanced it on the partially open door of the guest bedroom, thinking that his father would take the guests' coats there to put them on the bed. The boy then went off to play at a friend's house. The boy's father decided to keep all the guests' coats in the hall closet instead of putting them in the guest bedroom. Later that evening, one of the guests mistakenly wandered into the guest bedroom while searching for a bathroom. As the guest opened the door, the bucket fell and hit his head, causing a severe cut that required a dozen stitches. If the guest sues the boy for negligence, the most likely result is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}