{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1b89bf9-24fc-57a7-9956-6bb84d40cf0b","task_key":"default--test--1019","task_revision_id":"1","upstream_id":"1019","short_description":"A plaintiff was driving her car when she stopped at a red light at an…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible, if the photograph correctly and accurately portrayed the accident scene.', 'admissible, if the photograph was taken by a professional photographer.', 'inadmissible, if the defendant objects, because the photograph was taken two years after the accident.', \\\"inadmissible, if the defendant objects, because the photograph does not show the damage to the plaintiff's car.\\\", \\\"inadmissible, if the defendant objects, because the photograph might influence the jury's opinion unfairly.\\\", 'admissible, if the plaintiff can demonstrate that the conditions at the scene of the accident were the same as when the photograph was taken.', 'admissible, if the plaintiff identifies the photograph at trial.', 'inadmissible, if the defendant objects, because the photograph was not taken immediately after the accident.', 'inadmissible, if the defendant objects, because the photograph is non-verbal hearsay.', 'admissible, if the plaintiff can prove the photograph has not been altered in any way.']\",\"question\":\"A plaintiff was driving her car when she stopped at a red light at an intersection. A defendant, who was behind her, did not see the red light and failed to stop. He crashed into the rear of the woman's car and pushed her onto the curb and into a tree. When the plaintiff got out of her car, she felt fine physically, but was emotionally upset when she saw that the hood and trunk of her car were severely damaged. The next day she received an estimate from the mechanic for $1,950 to repair her car. She consulted an attorney the following afternoon and, that evening, suddenly developed shooting back pains. The plaintiff filed a lawsuit against the defendant based on a theory of negligence. One day prior to trial, two years after the accident, the plaintiff went to take a photograph of the accident scene. At trial, the photograph is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}