{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ecf29a9f-6909-5020-853d-741e49ee55e0","task_key":"all--test--ecf29a9f-6909-5020-853d-741e49ee55e0","task_revision_id":"1","upstream_id":"","short_description":"In which of the following situations is the defendant most likely guilty of…","config":"all","split":"test","body":"{\"choices\":\"['A defendant was a guide on a nature walk in a national park. The group was traversing a mountainous path when a mountain lion suddenly appeared. The defendant, who was carrying a loaded gun, knew that mountain lions and coyotes were in the area. The defendant saw that the mountain lion was about to attack one of the hikers. Although the defendant could have easily shot and killed it, he did nothing. The wild animal pounced on the hiker. As she was being savaged, the defendant and the other members of the group ran to safety.', 'A defendant, a registered nurse, asked her friend out to lunch. While eating, the friend suddenly began choking on a chicken bone. The defendant did nothing to help her friend. The friend choked to death.', \\\"A defendant, a physician, was walking home from his office one afternoon when he saw a car hit a man who was trying to cross the street. The victim was knocked to the ground and seriously injured, while the car sped away. The defendant, who could have saved the man's life if he had treated him, continued walking home without rendering assistance. The man died from loss of blood.\\\", \\\"A defendant took his four-year-old daughter out in the yard to play. As they were playing catch, the telephone began to ring. The defendant ran into the house to answer the phone. While he was inside, the daughter's ball rolled into the street. As she went to retrieve it, the daughter was struck and killed by a car.\\\"]\",\"question\":\"In which of the following situations is the defendant most likely guilty of manslaughter?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}