{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99225954-b7c8-5b0f-aba1-ff98943ad0d0","task_key":"all--test--99225954-b7c8-5b0f-aba1-ff98943ad0d0","task_revision_id":"1","upstream_id":"","short_description":"A husband and his passenger were killed in an auto collision involving a…","config":"all","split":"test","body":"{\"choices\":\"['objectionable, because the question is beyond the scope of direct examination.', 'objectionable, because it was highly prejudicial.', \\\"unobjectionable, because plaintiff's knowledge of such an affair was probative of the value to be placed on her loss.\\\", 'unobjectionable, because it was relevant on the issue of negligence.']\",\"question\":\"A husband and his passenger were killed in an auto collision involving a defendant. The husband's wife now brings an action to recover damages for loss of society and companionship. The wife testified that she and her husband had a close relationship and that they were happily married for over 15 years. Then on cross-examination, defendant's counsel asked her if she \\\"was aware of the affair that her husband was having with his passenger?\\\"The wife's attorney immediately objected to this question. The judge should rule that the question is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}