{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"177b3c12-0ce9-5fed-baa5-cec4fa11ae71","task_key":"all--test--177b3c12-0ce9-5fed-baa5-cec4fa11ae71","task_revision_id":"1","upstream_id":"","short_description":"A salesman, who had worked 20 years for the same company, was suddenly…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because the employee was present during the board meeting.', \\\"admissible, because the president's statement was an admission by a company representative.\\\", 'inadmissible, because the tape of the meeting is the best evidence.', \\\"inadmissible, because the secretary's transcribed notes are the best evidence.\\\"]\",\"question\":\"A salesman, who had worked 20 years for the same company, was suddenly terminated for no apparent reason. Thereafter, the salesman sued the company, alleging age discrimination. At trial, he wants to call an employee of the company as an adverse witness. The salesman seeks to have the employee testify that she was present at a company board meeting when the company's president allegedly said, \\\"Now, I'm sure that everyone agrees that the salesman is too old, and he really doesn't typify the image we want our employees to project. \\\" It is the common practice of the company to tape record all such board meetings. Moreover, it is customary for the company's secretary to transcribe the tapes following the board meetings. Upon objection by the company's attomey, the employee's proposed testimony will be held\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}