{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c63992c-1290-5c32-9345-b5f9688f7874","task_key":"all--test--8c63992c-1290-5c32-9345-b5f9688f7874","task_revision_id":"1","upstream_id":"","short_description":"A female employee was fired as an executive assistant at a time when she was…","config":"all","split":"test","body":"{\"choices\":\"['No, because the statement is clearly hearsay with no exception to the hearsay rule being applicable.', 'No, because it was based on things that the co-worker heard from others, making double-hearsay that is never admissible.', 'Yes, because that kind of testimony is reliable and not excludable as hearsay.', 'Yes, because it was admissible under the \\\"spontaneous utterance\\\" exception to the hearsay rules.']\",\"question\":\"A female employee was fired as an executive assistant at a time when she was seven months pregnant. She brought a Title VII civil rights action in federal court for gender discrimination against the employer. At trial, a former co-worker of the woman was not allowed to testify that she heard the human relations manager saying in an informal meeting that the plaintiff and her predecessor were both fired due to what the manager described as \\\"pregnancy complications.\\\" The proposed witness had been a vice-president of the company and a close adviser to the president. Without the benefit of that vital testimony, the jury entered a verdict for the defendant employer. On appeal, the plaintiff argued that the testimony should have been allowed and that it was not hearsay under the federal rules. Will the court likely grant the appeal, and order a new trial?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}