{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bac15184-9020-5949-9f03-91c35862bb52","task_key":"all--test--bac15184-9020-5949-9f03-91c35862bb52","task_revision_id":"1","upstream_id":"","short_description":"A 17-year-old boy was jostling with his 12-year-old brother. A pedestrian…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because the pedestrian reasonably believed that the boy was about to injure his brother.', 'prevail, based upon the doctrine of transferred intent.', 'not prevail, because the pedestrian assumed the risk.', 'not prevail, unless the boy was negligent.']\",\"question\":\"A 17-year-old boy was jostling with his 12-year-old brother. A pedestrian witnessed the boy shoving his brother and mistakenly believed that they were fighting. Acting as a Good Samaritan, the pedestrian approached the boys and tried to intercede. The boy stumbled and accidentally fell into the pedestrian, knocking him to the ground. The pedestrian broke his arm as a result of the fall. If the pedestrian sues the boy to recover damages for his injury, he should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}