Reducing Harm to Students From Deepfakes
The increased availability and usage of artificial intelligence (AI) creates continuous challenges for schools. Eleven percent of American children ages 9 to 17 know of a peer who has used AI to generate sexually explicit images of other kids, according a recent report. Protect your institution from new forms of online sexual harassment and be ready to properly handle allegations if they arise.
What Is a “Deepfake”?
Readily accessible AI image and video generators allow users to modify real images and videos or create “fully synthetic” images (wholly created by AI) depicting sexually explicit content or “deepfakes.” Deepfakes created with AI can look realistic and be difficult to distinguish between real and fake content. If the resulting photos or videos are of minors, they are child sexual abuse material and pose significant harms to the minors depicted in the deepfake.
Along with this technology, social media and encrypted messaging apps make it easier for students to circulate sexually explicit, nonconsensual intimate images (NCII) or material.
Many schools remain ill equipped to address deepfake incidents. Studies show fewer than 20% of high schoolers received any information from their school about deepfakes, and they learned even less about why sharing such images is harmful or where to report them. Students may be both perpetrators and victims; most victims are female. Parents are significantly less aware than students of these threats.
Legal Prohibition
Recognizing the issue’s severity, 49 states have enacted civil and criminal penalties for those engaging in this type of conduct, with many considering expanding the penalties to minors and first-time offenders.
Federal regulations specify that NCII, both authentic and deepfake, can create a hostile environment. Schools must use the same factors as when determining whether in-person conduct creates a hostile environment. Schools that receive federal funding also must address sexual harassment under Title IX.
Educate Your School Community
Combating the creation and sharing of deepfakes requires educating students, parents, teachers, and staff.
Students may not fully understand why creating and sharing deepfakes is harmful, or the potential ramifications for doing so.
Educate parents about the dangers of deepfakes and help them understand your policymaking process and why you ban deepfakes. Consider including their perspective when developing your comprehensive response.
Consider addressing, in an age-appropriate way, deepfakes and NCII in your curriculum as an educational preventative measure. Include the subject in sexual harassment prevention and digital citizenship efforts. Create and maintain current resources regarding NCII, including how to have the content removed from online platforms, how to request supportive measures in school, and where to seek counseling.
Teachers should understand what constitutes NCII and deepfakes and how to make a report if they learn a student has created or been a victim of a deepfake incident. All staff also should know how to make a report.
Train your Title IX Coordinator to respond to deepfake allegations, including if the conduct must be reported to law enforcement, how to protect student privacy, and how to take a victim-centered approach to investigations as well as supporting the person depicted in NCII.
Create a Policy: Prevent and Respond to a Deepfake Incident
Many schools lack policies and procedures that proactively address the sharing of deepfakes and only react once an incident occurs.
It is important to either:
- Create a standalone policy addressing deepfakes and NCII.
- Revise a preexisting policy — such your harassment policy or Title IX policy — to specifically address deepfakes and NCII.
- Any policy should highlight the potential for harm and provide a definition for deepfakes. In addition, any new or updated Title IX or sexual harassment policies should explicitly include online conduct — including creating or disseminating deepfakes or NCII — that creates a hostile environment.
Your policy also should include:
- Detailed information on where and how a student can confidentially report the incident to the school
- How your school will provide victim support through counseling, referral to external resources, or even help with removing the damaging content from online platforms
- Whether your school will help the victim in reporting the incident to law enforcement or be required to report to law enforcement even without the victim’s consent
If the creator of the NCII is also a student, follow your established procedures for handling allegations of sexual harassment. Consider what types of responses your institution may take to a deepfake incident involving students and include potential consequences for creating and distributing NCII in your policy.
Your policies should be meaningfully communicated to teachers, students, and parents.
More From UE
Youth Protection K-12 Resource Collection
Title IX and VAWA-Campus SaVE Act Resource Collection
Additional Resources
National Center for Missing & Exploited Children: Take it Down
About the Author
-
Lindsey Dunn
Senior Risk Management Counsel
Lindsey joined UE's Risk Management department in September 2024. Prior to that, she spent about six years as a Resolutions Counsel in the South Region for the Specialty Group. Before UE, Lindsey practiced labor and employment law. She is admitted to practice law in Florida and before the U.S. District Courts in Florida and the U.S. Court of Appeals for the Eleventh Circuit.