{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e437036-8027-59da-a8aa-ea46a417135e","task_key":"all--test--2e437036-8027-59da-a8aa-ea46a417135e","task_revision_id":"1","upstream_id":"","short_description":"A pedestrian was walking down the sidewalk past a building that was being…","config":"all","split":"test","body":"{\"choices\":\"['The pedestrian must prove beyond all reasonable doubt that the painting company was negligent.', 'The pedestrian must prove by a preponderance of the evidence that the painting company was negligent.', 'The painting company must prove by a preponderance of the evidence that it was not negligent.', 'There is a rebuttal presumption of negligence on the part of the painting company under the doctrine of res ipsa loquitur.']\",\"question\":\"A pedestrian was walking down the sidewalk past a building that was being painted. A ladder used by the painters fell and struck the pedestrian, causing an injury to his shoulder. In an action by the pedestrian against the painting company based on the theory of negligence, which of the following is the most accurate statement regarding the burden of proof?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}