{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"597cd6a2-182d-5dfd-9a19-0c7abfcc6381","task_key":"all--test--597cd6a2-182d-5dfd-9a19-0c7abfcc6381","task_revision_id":"1","upstream_id":"","short_description":"A victim was killed by a gunshot in the presence of three men. The fatal wound…","config":"all","split":"test","body":"{\"choices\":\"['Testimony that the witness is a cocaine addict.', 'Testimony that the witness embezzled money from his employer last year.', \\\"Testimony that the witness is not, in the neighbor's opinion, worthy of belief.\\\", 'Testimony that the witness is an atheist.']\",\"question\":\"A victim was killed by a gunshot in the presence of three men. The fatal wound was inflicted by a single shot, and there was no suggestion of a conspiracy among those present. One of the three men was charged with the killing. The prosecution called one of the other men as its first witness. His testimony, if true, established that the defendant fired the shot that killed the victim. The defendant's attorney then called the man's longtime neighbor to impeach him. Which of the following offers of evidence is most likely to be admitted for the impeachment of the man?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}