{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8cbdad06-e26d-5f97-bb4a-66bb8dacd1c0","task_key":"all--test--8cbdad06-e26d-5f97-bb4a-66bb8dacd1c0","task_revision_id":"1","upstream_id":"","short_description":"A 15-year-old boy was killed during a gang fight. Two days after his funeral,…","config":"all","split":"test","body":"{\"choices\":\"['No, because a person has no right to privacy after his or her death.', 'No, because the street was open to the public and the subject was newsworthy.', 'Yes, because the mother did not give permission to have the video used in the program.', 'Yes, because the mother suffered severe emotional distress as a result of viewing the video.']\",\"question\":\"A 15-year-old boy was killed during a gang fight. Two days after his funeral, the boy's mother saw a television program about gang violence and was shocked to see video of herself weeping over the boy's body. The video had been shot by the television reporting team while the boy's body was still lying on a public street. The mother suffered severe emotional distress as a result of seeing the video. If the mother sues the television station for invasion of her privacy and that of her son, will the mother be likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}