{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68bf2e90-0dc7-5d78-8993-4c2d93a7276b","task_key":"all--test--68bf2e90-0dc7-5d78-8993-4c2d93a7276b","task_revision_id":"1","upstream_id":"","short_description":"A woman sued a man for personal injuries that she suffered when she was struck…","config":"all","split":"test","body":"{\"choices\":\"['admissible as an opinion.', 'admissible as an admission.', 'admissible as a present sense impression.', 'admissible as a declaration against interest.']\",\"question\":\"A woman sued a man for personal injuries that she suffered when she was struck by the man's car. The man's car hit the woman as she was walking across the street. Immediately after the accident, the man ran over to the woman and said, \\\"I know I was driving fast, but you weren't paying attention where you were walking. Anyhow, I'm willing to pay all your medical expenses. \\\"At trial, the woman calls an eyewitness to the accident. The eyewitness proposes to testify that he heard the man tell the woman, \\\"I know I was driving fast. \\\" The man's attorney objects. If the eyewitness's testimony is admitted, it will most likely be because the proffered evidence is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}