{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42841f1a-8d36-53ab-89c2-465d7153ef02","task_key":"all--test--42841f1a-8d36-53ab-89c2-465d7153ef02","task_revision_id":"1","upstream_id":"","short_description":"A defendant was charged with vehicular battery after driving through a red light…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because it is irrelevant that this is not a homicide case.', 'admissible, because the woman believed her death was imminent when she made the declaration.', 'inadmissible, because of the close relationship between the woman and the passenger.', 'inadmissible, because it is hearsay not within any recognized exception.']\",\"question\":\"A defendant was charged with vehicular battery after driving through a red light and crashing into a woman's small convertible. The woman suffered massive internal injuries and lapsed into a coma for several hours after she reached the hospital. The woman's best friend, who was a passenger in the car, miraculously received only a few minor bruises. After the collision, the passenger stayed with the woman trying to comfort her until the ambulance arrived. At trial, the passenger is called to testify. €˜While on the witness stand, she gazes with sorrow at the woman, who is still wearing a cast on her leg. The passenger testifies that the first thing she remembered seeing after the accident was the woman's bloodied body trapped in the mangled wreckage. The state's attorney then asked the passenger if the woman said anything to her before being taken to the hospital. The passenger answered, \\\"Yes, she told me: €˜I know I'm dying. Why did he have to run that red light?\\\"Upon objection by the defendant's attorney, the court should rule the passenger's testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}