{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5085b86-bc76-5f7e-993f-9b735c12df56","task_key":"all--test--e5085b86-bc76-5f7e-993f-9b735c12df56","task_revision_id":"1","upstream_id":"","short_description":"A 13 year old boy was walking home from school. On the sidewalk, he was…","config":"all","split":"test","body":"{\"choices\":\"['No, because a butter knife can be as deadly as any other knife for the purpose of inflicting bodily harm, and the boy was conservative in his response to the situation.', 'No, because a person of that age in that factual situation could have reasonably been placed in fear of immediate bodily harm, and thus be entitled to act in his own defense.', 'Yes, because the boy had no right to react when he knew or should have known that someone could not be filleted with a butter knife.', 'Yes, because the response was disproportionate to the threat.']\",\"question\":\"A 13 year old boy was walking home from school. On the sidewalk, he was confronted and threatened by a stranger - a large bald man covered with tattoos €” who was holding a serrated butter knife. The man came close to the boy's face and in an angry voice said \\\"I am going to fillet you like a fish with this butter knife,\\\" and gestured in a menacing fashion. The boy took his army knife out, stabbed the man in the leg, and ran. The police officer who reviewed the matter decided that the man's threat was not objectively credible and that the boy had no authority to react. She arrested the boy for assault with a deadly weapon. The trial judge instructed the jury that the knife was a simple household utensil and not a deadly weapon. The boy was convicted at trial. Will the appellate court likely uphold the conviction?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}