{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b048908e-ac91-50a2-a14f-26b013d0c666","task_key":"all--test--b048908e-ac91-50a2-a14f-26b013d0c666","task_revision_id":"1","upstream_id":"","short_description":"A mother hated a girl because she always seemed to outperform her daughter and…","config":"all","split":"test","body":"{\"choices\":\"['Conspiracy and voluntary manslaughter.', 'Conspiracy, attempted murder, and two counts of murder.', 'Solicitation and two counts of murder.', 'Solicitation and attempted murder.']\",\"question\":\"A mother hated a girl because she always seemed to outperform her daughter and make her feel inferior. Fearing that the girl would beat out her daughter for the last cheerleading position, the mother decided to kill the girl. One night while the mother and her boyfriend were having dinner, the mother asked him to kill the girl. The mother handed her boyfriend a gun and gave him the home address where the girl lived. Unknown to the boyfriend, the mother gave him the wrong address. By mistake, the boyfriend went to the home of a family who lived across the street from the girl. The boyfriend rang the doorbell and a woman opened the door. The boyfriend asked the woman if her daughter was home. The woman called for her daughter to come to the front door. When she did so, the boyfriend shot and killed both the woman and her daughter. The mother should be found guilty for which, if any, of the following crimes?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}