{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f41e3a38-a45a-51c0-b172-4016d1bd55db","task_key":"all--test--f41e3a38-a45a-51c0-b172-4016d1bd55db","task_revision_id":"1","upstream_id":"","short_description":"In which of the following situations is the defendant's conduct most likely to…","config":"all","split":"test","body":"{\"choices\":\"['A defendant shot a victim in the head. The victim was then taken to a hospital for treatment of the wound. An earthquake later struck the city, causing the hospital to collapse. The victim was crushed to death in the rubble.', \\\"A defendant and a victim lived in the same apartment building. The defendant knew that the victim was having a love affair with a married woman. One day, the defendant learned that the victim was to be murdered by the married woman's husband. Although the defendant had ample time to warn the victim, he failed to do so. That night, the victim was stabbed to death by the husband.\\\", \\\"A victim, who was a gambler, was heavily in debt to the mob. The defendant, who was a mob enforcer, threatened to kill the victim if he didn't pay up. Frightened, the victim boarded the next airplane flight out of town. The airplane crashed, and the victim was killed.\\\", 'A defendant and a victim were driving to work together when the defendant, without provocation, stabbed the victim in the arm. The defendant then pushed him out of the car. The victim fell along the side of the street and fractured his ankle. Unable to move from the roadway, the victim was fatally crushed a half-hour later by a car driven by a drunk driver.']\",\"question\":\"In which of the following situations is the defendant's conduct most likely to make him criminally responsible for the victim's death?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}