{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1baaf088-1638-576c-b961-2908d7d1cc60","task_key":"all--test--1baaf088-1638-576c-b961-2908d7d1cc60","task_revision_id":"1","upstream_id":"","short_description":"A bank was robbed by a person wearing a yellow ski mask, goggles, and a…","config":"all","split":"test","body":"{\"choices\":\"['admissible as circumstantial evidence that the defendant committed the crime as charged.', 'admissible, to show the identity of the defendant.', 'inadmissible, because the probative value of the evidence is outweighed by the danger of unfair prejudice.', 'inadmissible, because it constitutes improper character evidence.']\",\"question\":\"A bank was robbed by a person wearing a yellow ski mask, goggles, and a fluorescent yellow jumpsuit. Three weeks after the robbery, a defendant was arrested and charged with the crime. At trial, the prosecuting attorney seeks to introduce evidence that the defendant had previously committed a robbery wearing a yellow ski mask, goggles, and a fluorescent yellow jumpsuit. Upon objection by the defendant's attorney, the proffered evidence should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}