This week, Sam Altman, CEO of OpenAI, and Arianna Huffington, founder and CEO of healthcare company Thrive Global, published article in Time persuading Develop artificial intelligencestartup backed by Thrive and OpenAI’s Startup Fund. The article suggests that AI could have a huge positive impact on public health by nudging people toward healthier behaviors.
Altman and Huffington write that Thrive AI is working on “a fully integrated AI personal trainer that provides real-time, personalized guidance and recommendations so you can take action based on your daily behaviors to improve your health.”
Their vision puts a positive spin on what could be one of AI’s sharpest twin blades. AI models are already adept at persuading humans, and we don’t know how much more powerful they could become as they grow and gain access to more personal data.
Alexander Madry, a professor on sabbatical from the Massachusetts Institute of Technology, leads a team at OpenAI called Preparedness that is working on this very question.
“One of the strands of work within Preparedness is persuasion,” Madry told WIRED in May. “It’s basically thinking about how these models can be used as a way to persuade people.”
Madry says he was drawn to OpenAI by the extraordinary potential of language models and the fact that the risks they pose have been barely explored. “There’s literally almost no science,” he says. “That was the impetus for Preparedness.”
Persuasiveness is a key element of programs like ChatGPT, and one of the ingredients that makes such chatbots so convincing. Language models are trained on human writing and dialogue, which includes myriad rhetorical and persuasive tricks and techniques. The models are also typically tuned to err on the side of statements that users find more persuasive.
Tests released In April, Anthropic, a competitor founded by OpenAI exiles, suggested that language models had gotten better at persuading people as they had grown in size and sophistication. Those studies involved giving volunteers a statement and then watching how an AI-generated argument changed their opinion of it.
OpenAI’s work involves analyzing AI in conversation with users — something that could unlock greater persuasive power. Madry says the work is being conducted on consenting volunteers and declines to disclose the findings so far. But he says the persuasive power of language models is profound. “As humans, we have this ‘weakness’ that if something is communicating to us in natural language, [we think of it as if] “It’s human,” he says, referring to the anthropomorphism that can make chatbots seem more realistic and convincing.
The Time article argues that the potential health benefits of persuasive AI will require sturdy legal safeguards because the models could have access to so much personal data. “Policymakers must create a regulatory environment that encourages AI innovation while protecting privacy,” Altman and Huffington write.
That’s not all policymakers will need to consider. It may also be crucial to consider how increasingly persuasive algorithms could be misused. AI algorithms could amplify disinformation or generate particularly persuasive phishing scams. They could also be used to advertise products.
Madry says a key question that hasn’t yet been explored by OpenAI or others is how much more persuasive or coercive AI programs that interact with users over time might be. Already, a number of companies offer chatbots that play the roles of romantic partners and other characters. AI girlfriends are increasingly popular—some are even designed to yell at you—but how addictive and persuasive these bots are is largely unknown.
The excitement and buzz surrounding ChatGPT following its November 2022 launch has led OpenAI, outside researchers, and many policymakers to focus on a more hypothetical question: could AI one day turn against its creators?
Madry says that risks ignoring the more subtle threats posed by silver-tongued algorithms. “I worry that they’re going to focus on the wrong questions,” Madry says of the work policymakers have done so far. “That in some sense, everyone’s saying, ‘Oh yeah, we’re dealing with this because we’re talking about it,’ when in fact we’re not talking about the right thing.”