{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"015f7f05-839d-5efc-8a86-c91cb667f17f","task_key":"all--test--015f7f05-839d-5efc-8a86-c91cb667f17f","task_revision_id":"1","upstream_id":"","short_description":"A pedestrian was injured when hit by a chair that was thrown from an upper-story…","config":"all","split":"test","body":"{\"choices\":\"['No, because it is unreasonable to expect the pedestrian to prove which of the defendants caused the harm.', 'No, because of the doctrine of alternative liability.', \\\"Yes, because a plaintiff always has the burden to prove that a particular defendant's conduct was the factual cause of the plaintiff's physical harm.\\\", 'Yes, because the pedestrian has failed to offer evidence that the defendants jointly engaged in tortious conduct.']\",\"question\":\"A pedestrian was injured when hit by a chair that was thrown from an upper-story hotel window. The pedestrian sued the occupants of all the rooms from which the chair might have been thrown. At trial, the pedestrian has been unable to offer any evidence as to the exact room from which the chair was thrown. The defendants have filed a motion for a directed verdict. Should the court grant the motion?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}