{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0b299cf-8c47-5b0c-8943-1ef5ba324ee4","task_key":"all--test--e0b299cf-8c47-5b0c-8943-1ef5ba324ee4","task_revision_id":"1","upstream_id":"","short_description":"A student at a private university sued the university in fed-eral court for…","config":"all","split":"test","body":"{\"choices\":\"['Abuse of discretion.', 'Clearly erroneous.', 'De novo.', 'Harmless error.']\",\"question\":\"A student at a private university sued the university in fed-eral court for negligence after he fell from scaffolding in a university-owned theater building. At trial, after briefing from both parties, the court permitted the jury to hear testimony that there had been several previous accidents in the same building. The jury found for the student, and the university appealed. One of the university's arguments on appeal is that the testimony about the previous accidents should have been excluded as irrelevant and highly prejudicial. Which standard of review applies to this argument?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}