{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e67220de-ee52-5fba-9ac6-542728c0e883","task_key":"all--test--e67220de-ee52-5fba-9ac6-542728c0e883","task_revision_id":"1","upstream_id":"","short_description":"A child was riding his bicycle on a street, heading from north to south. As he…","config":"all","split":"test","body":"{\"choices\":\"['prevail, if the child would have been injured even more severely if he had crashed his bike into the car, rather than falling off the bike.', \\\"prevail, because the child's action in trying to stop the bike was foreseeable.\\\", 'not prevail, because the driver and his car never touched the child or bike.', 'not prevail, unless the driver negligently created a risk of harm that caused the child to fall off the bike.']\",\"question\":\"A child was riding his bicycle on a street, heading from north to south. As he went down a hill, he came to an intersection. At that moment, a car was coming toward the intersection, heading from east to west. When the driver saw the child on the bike, he slammed on his brakes in an attempt to avoid hitting the child. Meanwhile, the child realized that he was in danger of colliding with the car, so he applied the hand brakes on his bike as hard as possible. As a result of trying to stop so suddenly, the child flew over the handlebars of the bike and onto the ground. Neither the child nor his bike touched the car, but the child suffered injuries from falling off the bike. If the child's parents, on behalf of the child, assert a claim against the driver to recover damages for the child's injuries, they will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}