{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f84f9ea-6d18-5275-94c0-0cc246b9873b","task_key":"all--test--7f84f9ea-6d18-5275-94c0-0cc246b9873b","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued a trucking company for injuries allegedly suffered when a…","config":"all","split":"test","body":"{\"choices\":\"['Testimony concerning subsequent repairs to the truck paid for by the trucking company where they have stipulated to ownership.', 'Color pictures of a plastic surgeon operating on the plaintiff showing the incision and bloody surgical tools.', \\\"Testimony from the plaintiff's boss concerning the amount of time she has missed work, when payroll records had already been admitted as evidence of the plaintiff's lost wages.\\\", 'Testimony of a surprise witness to the accident when the court is willing to grant the trucking company a continuance to prepare for crossexamination.']\",\"question\":\"A plaintiff sued a trucking company for injuries allegedly suffered when a speeding truck jackknifed and struck her car. Which of the following pieces of evidence, if offered by the plaintiff, is most likely to be admitted by the court?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}