“When we look at intimate image abuse, the vast majority of tools and weaponization come from the open source space,” Ajder says. But they often start with well-meaning developers, he says. “Someone creates something they think is interesting or cool, and someone with bad intentions sees its harmful potential and uses it as a weapon.”
Some, such as the repository disabled in August, have communities around them specifically set up for explicit uses. The model positions itself as a tool for deepfake porn, Ajder claims, becoming a “pathway” for abuse that mainly targets women.
Other videos uploaded to the porn streaming site using an account hosting AI models downloaded from GitHub featured the faces of popular deepfake targets, celebrities Emma Watson, Taylor Swift and Anya Taylor-Joy, as well as other less notable but very real women , imposed on sexual situations.
The developers freely described the tools they used, including two searched by GitHub but whose code survives in other existing repositories.
Perpetrators hunting deepfakes congregate in many places across the Internet, including secret community forums on Discord and prominently on Reddit, hampering efforts to prevent deepfakes. One Redditor offered his services using the archived repository software on September 29. “Can someone do this to my cousin,” asked another.
Torrents from the main repository, which were banned by GitHub in August, are also available in other corners of the web, showing how challenging it is to universally control counterfeit open source software. Other deepfake porn tools, such as the DeepNude app, have already done this similarly removed before modern versions appeared.
“There are so many models, so many different forks of models, so many different versions, that it can be difficult to keep track of them all,” says Elizabeth Seger, director of digital policy at the cross-party British think tank Demos. “Once a model is publicly available for download in an open source format, there is no way to publicly withdraw it,” he adds.
One of the deepfake porn creators who recorded 13 manipulated videos of celebrities has admitted that one of GitHub’s top repositories is advertised as a “NSFW” version of another project, encouraging responsible apply and explicitly asking users not to apply it for depictions nudity. “I’m learning all the Face Swap AI available from GitHUB, and I’m not using online services,” their profile on the channel site says cheekily.
GitHub had already disabled this NSFW version when WIRED identified the deepfake videos. However, on January 10, other repositories marked as “unlocked” versions of the model were available on the platform, including one with 2,500 “stars”.
“Technically it is true that once [a model is] It can’t be reversed there. But we can still make it difficult for people to access,” Seger says.
He adds that counterfeit “porn” can cause more than just psychological damage if left unchecked. Its knock-on effects include the intimidation and manipulation of women, minorities and politicians, as can be seen with the program political deepfakes influences women politicians around the world.
But it’s not too behind schedule to get the problem under control, and platforms like GitHub have options, Seger says, including intervening at the point of submission. “If you put a model on GitHub and GitHub says no, and all the hosting platforms say no, it will be harder for the average person to get that model.”
Stopping deepfake porn created using open source models also depends on policymakers, tech companies, developers, and, of course, the creators of the offensive content themselves.
According to the nonprofit group Public Citizen’s, at least 30 U.S. states have some kind of counterfeit porn laws, including bans. Legislative trackeralthough definitions and rules vary, and some laws only apply to minors. Deepfake creators in the UK will also soon feel the force of the law following the government’s announcement criminalizing the creation of sexually explicit deepfakesand making them available on January 7.