{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d62a7b37-e41e-5760-9448-2b4ccb5da6e6","task_key":"all--test--d62a7b37-e41e-5760-9448-2b4ccb5da6e6","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued an industrial facility in her neighborhood for injuries to her…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because reviewing it did not refresh the plaintiff's recollection.\\\", 'No, unless it is offered by the defendant.', 'Yes, and the plaintiff should be allowed the option of reading it into evidence or having the diary received as an exhibit.', 'Yes, and the plaintiff should be allowed to read the diary into evidence.']\",\"question\":\"A plaintiff sued an industrial facility in her neighborhood for injuries to her health caused by air pollution. At trial, the plaintiff was asked questions on direct examination about the days on which she had observed large amounts of dust in the air and how long the condition had lasted. She testified that she could not remember the specific times, but that she maintained a diary in which she had accurately recorded this information on a daily basis. When her attorney sought to refresh her recollection with her diary, she still could not remember. The plaintiff's attorney seeks to have the information in the diary admitted at trial. Is the information admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}