{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2802f39d-120c-5526-b9f6-331ea564e53e","task_key":"all--test--2802f39d-120c-5526-b9f6-331ea564e53e","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued a defendant in connection with the dissolution of a partnership…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the business attorney's professional relationship with the plaintiff and the defendant has ended.\\\", 'No, because the plaintiff and the defendant consulted the business attorney jointly.', 'Yes, because either the plaintiff or the defendant may block disclosure of statements made during such meetings.', 'Yes, because either the plaintiff or the defendant may claim the privilege on behalf of the partnership.']\",\"question\":\"A plaintiff sued a defendant in connection with the dissolution of a partnership they had formed to run a parcel delivery service. They had relied on a business attorney in establishing the business. After the business failed, the plaintiff and the defendant disagreed about their respective obligations. At trial, both have hired new counsel. The plaintiff calls the business attorney to testify to representations the defendant made in meetings she had with the plaintiff and the business attorney. The defendant objects to the business attorney's testimony, invoking the attorney-client privilege. Should the court uphold the defendant's privilege claim?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}