{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c25718b-707d-5983-b787-75bfb6cc4915","task_key":"all--test--0c25718b-707d-5983-b787-75bfb6cc4915","task_revision_id":"1","upstream_id":"","short_description":"A defendant was arrested and charged with burglarizing a home. At trial, the…","config":"all","split":"test","body":"{\"choices\":\"['admitted without restriction as probative evidence of motive.', \\\"admitted, provided the court instructs the jury to limit use of the evidence to the defendant's motive to burglarize.\\\", 'excluded as lacking any probative value.', 'excluded as substantially more unfairly prejudicial than probative.']\",\"question\":\"A defendant was arrested and charged with burglarizing a home. At trial, the main issue of contention centered on the identity of the burglar. The state called the homeowner to testify, but his identification testimony was inconclusive. The homeowner testified that he was asleep during the burglary and awoke as the burglar was fleeing the premises. The homeowner admitted that he didn't see the face of the burglar and couldn't even tell whether the person was male or female. Thereafter, the prosecution offers evidence that at the time of the burglary, the defendant needed money to defend himself against other burglary charges. This proffered evidence should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}