{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8114fae7-9d0b-5071-ae0f-e49d1eab083e","task_key":"all--test--8114fae7-9d0b-5071-ae0f-e49d1eab083e","task_revision_id":"1","upstream_id":"","short_description":"A wealthy woman often wore expensive jewelry while walking her dog in the park.…","config":"all","split":"test","body":"{\"choices\":\"['Battery.', 'Assault.', 'Robbery.', 'No crime.']\",\"question\":\"A wealthy woman often wore expensive jewelry while walking her dog in the park. Her friends warned her against wearing such valuable jewelry because they feared she would be an easy target for muggers. In order to persuade the woman not to wear her expensive jewelry in the park, her friend decided to play a practical joke. One morning, the friend dressed like a man and hid in an area of the park that she knew the woman customarily walked through. As the woman was strolling through the park with her dog that morning, the friend jumped out from behind the bush brandishing a toy pistol and grabbed the woman's diamond necklace from her neck. Startled, the woman became hysterical and began to plead for her life. The friend then removed her male garb, handed the necklace back to the woman and said, \\\"I just wanted to frighten you to teach you a lesson. \\\"If the friend is subsequently prosecuted, she should be found guilty of which, if any, of the following crimes?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}