{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fc6b8f7-d4f3-5f89-a92d-71f20c9213a1","task_key":"all--test--1fc6b8f7-d4f3-5f89-a92d-71f20c9213a1","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff was driving her car when she stopped at a red light at an…","config":"all","split":"test","body":"{\"choices\":\"['inadmissible, if the defendant objects, because the photograph is non-verbal hearsay.', 'inadmissible, if the defendant objects, because the photograph was taken two years after the accident.', 'admissible, if the plaintiff identifies the photograph at trial.', 'admissible, if the photograph correctly and accurately portrayed the accident scene.']\",\"question\":\"A plaintiff was driving her car when she stopped at a red light at an intersection. A defendant, who was behind her, did not see the red light and failed to stop. He crashed into the rear of the woman's car and pushed her onto the curb and into a tree. When the plaintiff got out of her car, she felt fine physically, but was emotionally upset when she saw that the hood and trunk of her car were severely damaged. The next day she received an estimate from the mechanic for $1,950 to repair her car. She consulted an attorney the following afternoon and, that evening, suddenly developed shooting back pains. The plaintiff filed a lawsuit against the defendant based on a theory of negligence. One day prior to trial, two years after the accident, the plaintiff went to take a photograph of the accident scene. At trial, the photograph is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}