Deepfakes are manipulated or synthetic audio or visual media that seem authentic, and which feature people that appear to say or do something they have never said or done, produced using artificial intelligence techniques, including machine learning and deep learning.
Deepfake production relies on an innovative deep learning technique called ‘generative adversarial networks’ (GANs), which can increase both the degree of automation and the quality of the output compared to conventional techniques. GANs generate deepfakes by pitting two AI agents – also described as artificial neural networks – against each other. While the producer agent learns to create fakes that look just like standard recordings, a detector agent learns to identify whether a media product is fake or authentic.
Deepfakes: promising potential, but abuse is inevitable
The basis for all deepfake software is an artificial intelligence algorithm called ‘deep learning’, which needs a massive amount of data to learn how to replace a face within a video. The number of readily available images of celebrities on the web is why they have quickly become the most prominent targets in the production of fake videos.
Hollywood actress Bella Thorne is one of the most deepfaked people in the world: videos and pictures of her have been virtually altered and edited into a vast number of other videos. All of these deepfakes were non-consensual, and most have been abusive. Hackers have used the deepfakes to harass and blackmail Thorne, who has reacted by releasing intimate footage of herself.
A 2020 study by digital researcher Sophie Maddocks shows that, in certain instances, deepfake creators deliberately use manipulated videos to silence famous personalities. Maddocks mentions that Thorne was harassed because she had spoken out against sexual violence. According to the study, the prevalence of pornographic deepfake videos is underrepresented in both media coverage and research because most outlets and researchers focus on the abuse of fake videos for political ends.
Despite their harmful potential, deepfakes are not malicious, per se. As parody versions of popular internet clips and proof-of-concept videos first began to pop up, the potential for the filmmaking industry became clear. In fact, Lucasfilm recently hired a Youtuber called ‘Shamook’, who, in an independent side project, replaced CGI-generated footage of famous Star Wars characters like Luke Skywalker with more realistic looking deepfake versions of themselves.
Malicious in use, hard to spot
Such benign applications, however, are the exception – not the rule. A 2019 Deeptrace Labs report investigated the 15 000 deepfake videos online at the time, a number that had doubled since nine months earlier. Remarkably, 96 % of these videos were pornographic and featured the swapped-in faces of women who had never given their consent. The report’s authors also estimated that 99 % of the doctored adult content utilised the faces of celebrities, particularly actresses such as Kristen Bell and Gal Gadot. The ESMH wrote also about this report back in 2019. What has changed since then?
Henry Ajder, Deepfake expert at Deeptrace: “If we fast forward to 2021, the landscape looks the same and different at the same time. It is different because deepfakes have become increasingly commoditised; that is, tools for creating novelty face swaps, fun and meaningful content have really exploded. But the malicious-use landscape has not significantly changed in terms of who is primarily being victimised.” – Read Henry Ajder’s full interview
Accessible to everyone, deepfakes move to unmoderated spaces
A plethora of ‘freemium’ apps like ‘ReFace’, which can replace the faces in any video at the tap of a finger, have surfaced on app stores, with the technology accessible to anyone at virtually no cost. ‘ReFace’ has registered more than 100 million downloads to date on the Google Play Store alone.
In September 2021, a new app specifically designed for swapping real faces into existing porn movies appeared on the web. After MIT Technology Review reported on it, the creators of the unnamed app withdrew it from the net. One reason that apps like this keep appearing is the relative ease with which code repositories detailing how to create deepfakes can be accessed. While established porn sites like PornHub and social media websites like Reddit have banned most deepfake-related content, the tools to create these videos circulate freely around the web.
Henry Ajder: “Current detection mechanisms are neither scalable nor robust enough to be reliably rolled out to major platforms like Facebook or Twitter. Facebook would have to use general computer vision algorithms to detect adult content. The platform regularly bans content that violates its terms of service. Achieving this would be much harder on Twitter, because the microblogging service specifically allows adult content.”
As a study by researchers at the University of Central Florida (UCF) shows, the code base for deepfake projects like face swaps is openly accessible on GitHub. The study’s authors compare the toxic geek masculinity involved in the deepfake community to some of the same ideals and characteristics that drive the open-source software community. They also identify telltale elements like ‘indifference to harm’, the ‘abstraction of human subjects’ and a ‘disassociation from porn’ on the GitHub forums. In a follow-up study, UCF researchers show that deepfake projects have successfully moved from moderated spaces to unmoderated spaces like GitHub, while still maintaining ties to Reddit.
Three categories of harm
The Panel for the Science and Future of Technology (STOA) has published a study that closely examines the impact of deepfakes and some possible meaningful regulatory actions that can be taken to address them. The study’s authors draw attention to three categories of harm: personal harm, including psychological damage and a loss of reputation; organisational and group harm, which involves the blackmailing of families and companies and a loss of trust in political parties; and finally, societal harm, namely the exacerbation of gender inequality and stereotypes and a growing climate of general mistrust.
How to tackle deepfakes: bans, labels and media literacy
The report’s findings directly support the forthcoming EU Digital Services Act and Artificial Intelligence Act, which aim to tackle deepfake regulation from five different angles: creation, circulation, technology, target and audience.
Mariëtte van Huijstee, Rathenau Institute: “In certain applications, we might consider banning deepfakes. A full ban would be disproportionate because of deepfakes’ beneficial applications in the media and for medical and educational applications, but for non-consensual pornography and political disinformation campaigns, a ban might be justified.” – Read Mariëtte van Huijstee’s full interview
“Labelling deepfakes is important as it could reduce the impact of perceived fake news. Social media platforms should be obliged to automatically label deepfakes as such, and uploaders should be able to contest the labelling”. When asked whether she thinks this approach would yield better results than the mixed results from the Code of Practice on Disinformation, van Huijstee says: “Yes, because instead of a voluntary commitment, the Digital Services Act consists of standardised procedures and obligations.”
What shocked her the most while researching the report, van Huijstee says, was the erosion of trust that individuals developed because of deepfakes. ‘Educating the audience will be crucial to fight the harms of deepfakes. This needs to start in primary education, where pupils should be equipped with the tools of journalists and fact-checkers, so they can judge the trustworthiness of a source.’
Henry Ajder: “It is important to educate the public but we must beware of a backfire effect. Informing the public in detail about deepfake technology could also drive them towards it. Creating a duty of care on generalised online harms is very important, and I am currently working on that issue with several UK governmental organisations.”
Such legal initiatives and the STOA report arrive at a time of urgent need for policies regulating malicious deepfake videos, since, in most countries, victims like Bella Thorne cannot currently expect to receive any kind of legal protection.
Related content
• A scientist’s opinion: Henry Ajder: “We need to make finding and accessing these tools as difficult as possible”
• A scientist’s opinion: Mariëtte van Huijstee: “Video footage is perceived as evidence”
