{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e60942fc-ef71-5db7-bced-7b9fe2aff92d","task_key":"all--test--e60942fc-ef71-5db7-bced-7b9fe2aff92d","task_revision_id":"1","upstream_id":"","short_description":"A young boy and a young girl were playing with a football on the sidewalk next…","config":"all","split":"test","body":"{\"choices\":\"['prevail, unless the driver was driving his vehicle within the posted speed limit.', 'prevail, unless the girl fell because the boy negligently pushed her toward the street.', 'not prevail, unless the driver was driving negligently when the accident occurred.', 'not prevail, unless the driver had the last clear chance to avoid the accident']\",\"question\":\"A young boy and a young girl were playing with a football on the sidewalk next to a street. They were laughing as they pushed and pulled each other, trying to get the ball and keep it away from each other. Just as a car approached on the street, the girl lost her balance and fell off the sidewalk and into the street. The driver of the car tried to swerve to avoid her, but ran over her leg. The girl suffered a broken leg. If the girl's parents assert a claim on her behalf against the driver to recover damages for her injuries, they will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}