{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11eca867-587b-5b8d-9a96-2b5fcc57f167","task_key":"all--test--11eca867-587b-5b8d-9a96-2b5fcc57f167","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff, a management trainee, brought a sex discrimination lawsuit against…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the janitor's statement is hearsay not within any exception.\\\", 'No, because the statements of both the janitor and the supervisor are hearsay not within any exception.', \\\"Yes, because the janitor's statement is a present sense impression, and the supervisor's statement is a statement of his then-existing state of mind.\\\", 'Yes, because the statements of both the janitor and the supervisor are statements janitor and the supervisor are statements concerning a matter within the scope of their employment.']\",\"question\":\"A plaintiff, a management trainee, brought a sex discrimination lawsuit against her employer for wrongful termination of her employment. At trial, the plaintiff is prepared to testify that a janitor at the company told her that he had heard her supervisor say to other male coworkers about her, \\\"Make it hard for her. Maybe she'll go home where she belongs.\\\" Is the plaintiff's proposed testimony admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}