{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80e77550-b3cc-5321-8a23-a897f819d792","task_key":"all--test--80e77550-b3cc-5321-8a23-a897f819d792","task_revision_id":"1","upstream_id":"","short_description":"A college student and a man got into a fight at a bar while playing pooi. The…","config":"all","split":"test","body":"{\"choices\":\"['admissible as part of the res gestae.', 'admissible as an admission.', 'inadmissible as hearsay not within any recognized exception.', 'inadmissible as self-serving.']\",\"question\":\"A college student and a man got into a fight at a bar while playing pooi. The college student allegedly started the fight when he struck the man in the head with a pool cue after the man sank a game-winning shot. At the college student's trial for assault, the prosecution called one of the college student's classmates to testify. The classmate was not at the bar during the fight. The classmate offered to testify that the next day at class, he approached the college student and said, \\\"It was wrong of you to start a fight with the man over a game of pool,\\\" at which point, the college student bowed his head and walked away. The classmate's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}