{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a21eeb18-5ac0-5afd-9b90-284dba1cde55","task_key":"all--test--a21eeb18-5ac0-5afd-9b90-284dba1cde55","task_revision_id":"1","upstream_id":"","short_description":"A detective found a footprint from a left-foot shoe at a murder scene. The print…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because it is direct evidence of guilt.', 'Yes, because it is appropriate circumstantial evidence of guilt.', 'No, because this shoe is irrelevant and will have no residue or other probative evidence relating to it.', 'No, because footprint evidence is highly speculative unless both shoes and shoe prints are offered and authenticated.']\",\"question\":\"A detective found a footprint from a left-foot shoe at a murder scene. The print was preserved appropriately as evidence. It had distinctive tread marks and an unusual wear pattern on the sole. It also had a \\\"V\\\" mark on the heel bottom that indicates the brand was a Victory shoe. The detective, armed with a proper search warrant, searched the suspect's apartment, where he found a shoe to a right foot that of the same size, and with a similar wear pattern and the same \\\"V\\\" mark as the shoeprint found at the scene. The shoe for the left foot was not found but the shoe for the right foot was seized from the suspect's closet and offered as evidence at trial. Is this admissible evidence?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}