They were published by a group of current and former OpenAI employees public letter warning that the company and its rivals are creating artificial intelligence with excessive risks, without sufficient oversight, and muzzling employees who may witness irresponsible actions.
“These threats include further perpetuation of existing inequalities, manipulation and disinformation, up to loss of control over autonomous artificial intelligence systems, which may result in the extinction of humanity,” we read in a letter published on the website Righttowarn.ai. “Until there is effective government oversight of these corporations, current and former employees are among the few people who can hold them accountable.”
The letter calls not only on OpenAI but on all AI companies to commit not to penalize employees who speak out about their activities. It also calls on companies to establish “verifiable” ways for employees to provide anonymous feedback on their performance. “Ordinary whistleblower protection is insufficient because it focuses on illegal activity, while many of the threats that concern us are not yet regulated,” the letter says. “Some of us are legitimately concerned about various forms of retaliation, given the history of such incidents in the industry.”
Last month, OpenAI faced criticism Vox article revealed that the company threatened to claw back employees’ equity unless they signed non-disparagement agreements that prohibited them from criticizing the company or even mentioning the existence of such an agreement. OpenAI CEO Sam Altman he said recently on X that he was unaware of such arrangements and the company never recovered anyone’s capital. Altman also said the clause would be removed, allowing employees to speak out. At the time of publication, OpenAI had not responded to a request for comment.
OpenAI has also recently changed its approach to security management. Last month, the OpenAI research group responsible for assessing and countering the long-term threats posed by the company’s more powerful artificial intelligence models was effectively disbanded after several prominent figures left and the remaining team members were absorbed into other groups. A few weeks later the company announced that it has created a Safety and Security Committee, headed by Altman and other board members.
Last November, Altman was fired by OpenAI’s board of directors for allegedly failing to disclose information and intentionally misleading it. After a very public row, Altman returned to the company and most of the board was ousted.
The letter’s signatories include people who have worked on security and governance at OpenAI, current employees who signed anonymously, and researchers who currently work at competing artificial intelligence companies. It has also been endorsed by several renowned artificial intelligence researchers, including Geoffrey Hinton and Yoshua Bengio, who both won the Turing Award for their pioneering research in artificial intelligence, and Stuart Russella leading expert in artificial intelligence security.
Former employees who signed the letter include William Saunders, Carroll Wainwright and Daniel Ziegler, all of whom worked on artificial intelligence security at OpenAI.
“Currently, the general public underestimates how fast this technology is developing,” he says Jacob Hilton, a researcher who previously worked on reinforcement learning at OpenAI and who left the company over a year ago to pursue recent research opportunities. Hilton says that while companies like OpenAI commit to building AI safely, there is little oversight to ensure this is the case. “The protections we are asking for are intended to apply to all pioneering AI companies, not just OpenAI,” he says.
“I left because I lost confidence that OpenAI would behave responsibly,” says Daniel Kokotajlo, a researcher who previously worked on artificial intelligence management at OpenAI. “Things happened that I think should have been disclosed to the public,” he adds, sinking to provide details.
Kokotajlo says the proposed letter would provide greater transparency and believes there is a good chance that OpenAI and other companies will reform their policies, given the negative reaction to news of the non-disparagement agreements. He also says that artificial intelligence is developing at an alarming speed. “The stakes are going to be much, much higher in the next few years,” he says, “at least that’s what I think.”