CEOs of companies several major artificial intelligence companies are pushing for members of Congress to pass fresh legislation that would make it harder for bad actors to develop biological weapons using their technology.
Demis Hassabis of Google DeepMind, Sam Altman of OpenAI, Dario Amodei of Anthropic and Mustafa Suleyman of Microsoft AI were among the signatories of the agreement public writing calling for legislation requiring companies selling synthetic DNA and RNA to screen customers and to prevent the misuse of genetic material.
The letter, organized by the nonpartisan Institute for Progress and the right-wing Foundation for American Innovation, acknowledged that given the pace of artificial intelligence development, “there is a real possibility that the knowledge barriers that have historically prevented bad actors from obtaining biological weapons will significantly erode.”
Scientist Arthur Kornberg was the first to successfully synthesize DNA in the 1950s. Today, the process is automated, with dozens of companies around the world using commercial synthesizers to “print” and sell custom genetic sequences used in scientific research, drug development and diagnostics. Many suppliers sell only to qualified researchers, biotechnology companies and educational institutions, but not all of them check customers or the gene sequences they order.
In 2017, Canadian researchers raised the alarm when they used $100,000 worth of mail-order DNA to recreate the extinct horsepox virus. Critics say the same methodology could be used to engineer smallpox, a closely related and deadly virus. Since then, gene synthesis has only become cheaper.
Combined with advances in artificial intelligence, it is now possible to design unsafe fresh toxins and pathogens using huge language models, although some biology training would likely still be needed to create a functional virus from scratch. Although bioterrorist attacks have been infrequent, they can cause mass casualties, social panic, and economic losses. A solemn concern is that an AI-designed pathogen could intentionally or unintentionally cause a global pandemic.
“AI tools enable the user to very quickly determine where to turn to order sequences that will not be subject to scrutiny,” says David Relman, a microbiologist and biosecurity expert at Stanford University who signed the letter. “If prompted properly, they can also tell you how to change the nature of the order so that even people checking it will have a much harder time detecting what you are trying to prepare.”
Signatories include other scientists, national security experts, and executives from gene synthesis companies Twist Bioscience and Ansa Biotechnologies. These companies are members of the International Gene Synthesis Consortium, established in 2009 to implement voluntary screening practices. Many companies already utilize software to screen orders for “sequences of concern” that may contribute to body toxicity or disease.
“If you have technology that can synthesize DNA, you want to make sure it’s used responsibly, and part of that is making sure you understand what you’re doing and who you’re doing it for,” says James Diggans, vice president of policy and biosecurity at Twist Bioscience. The company has been supporting the implementation of formal rules for years.
Federal guidelines introduced under the Biden administration required scientists and companies receiving federal funds to order synthetic gene sequences from suppliers that monitor purchases. AND bipartisan bill introduced earlier this year in the Senate would require all gene synthesis providers operating in the U.S. to screen orders and customers for bad actors or unsafe pathogens.
However, screening tools are not perfect. Last year, Microsoft researchers published a report entitled test showing that AI protein design tools were able to generate potentially unsafe gene sequences that slipped through the companies’ screening software. The models suggested fresh protein sequences with structures similar to those known to be unsafe.
Geoff Ralston, former president of Y Combinator and partner at the Unthreatening AI Fund, believes that AI labs with biological models should self-screen users.
“It should be very difficult, if not impossible, to ask a model to help you do something immediately dangerous,” says Ralston, who also signed the letter.
Relman agrees that regulating audit procedures is only part of the solution. “Given that in some cases the inspection may fail, we need to have other inspection points,” he says. “This is where AI companies will need to step up.”