{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d4bac55-3a9d-55c7-a177-e8850cbce960","task_key":"all--test--3d4bac55-3a9d-55c7-a177-e8850cbce960","task_revision_id":"1","upstream_id":"","short_description":"A victim was standing on a street corner waiting to make a drug buy. The drug…","config":"all","split":"test","body":"{\"choices\":\"['the jury, after a preliminary determination by the judge.', 'the jury, without a preliminary determination by the judge.', 'the judge, with instruction to the jury that it may consider the dying declaration if it determines that the victim knew he was dying.', 'the judge, without assistance from the jury.']\",\"question\":\"A victim was standing on a street corner waiting to make a drug buy. The drug dealer was always late, so the victim continued to wait outside in the freezing cold weather because he needed a quick fix. Suddenly a car rounded the street corner and a shot rang out. The victim slumped over and fell to the sidewalk. Immediately after being shot and before he died, the victim exclaimed, \\\"The defendant did it!\\\" The defendant was indicted and is now on trial for the victim's murder. Based on these facts alone, the admissibility of the victim's statement should be determined by\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}