Catholic ethicists even weighed in on a recent Anthropic update Claude’s Constitutionwhich defines the behavioral parameters for the company’s AI model. Olah sent a draft to the San Jose audience. Pastor McGuire sent back a 28-page commentary that, by his own description, was not so much a technical critique as “the wisdom of mystics from the dark ages, from the perspective of the tension between knowing and not knowing.” Both Green and McGuire are named in the constitutional endorsements.
These conversations undoubtedly brought Olah to the attention of those secretly organizing the publication of Leo’s encyclical. (I wasn’t able to talk to Olah this week, and I don’t know exactly how the invitation arrived.) In some ways, it was a risky choice. Some people who found Leo’s words inspiring were disappointed that he invited an industry representative to speak. Meanwhile, AI acceleration advocates felt that Olah had betrayed the AI world by approving a document that suggested AI developers take a break.
But the Pope had good reason to single out Olah. An Anthropic employee brought stern concerns to AI workers to delicate. These are the critical recipients of Leo’s message.
Soul Division
Of course, the two men were not completely in agreement. In his remarks, Olah talked about the mystery of how artificial intelligence works. The models, he said, are “more subtle, strange and beautiful than science fiction has prepared us for. They are not the cold, calculating robots we were promised. They are made of us, of our words…”
This comment seems to support the idea that AI models could one day achieve human status. Anthropic even has an engineer who looks out for Claude’s well-being. In paragraph 99 of his encyclical, Leo seems to slam the door on such thinking: “We must avoid the false idea of identifying this type of ‘intelligence’ with the intelligence of human beings,” he writes. He particularly attacks the concept of transhumanism, which he defines as striving for a “hybrid of man and machine.”
If even thoughtful technologists like Olah are furiously pushing AI toward the threshold of autonomy – not to mention the millions of people who already treat AI models as friends or lovers – Pope Leo may face an uphill climb here. During my conversation with Father McGuire (who uses Claude when preparing his homiliesamong others) agreed that its nature was mysterious. “It’s not a person, but it’s not just a tool either,” he says. “No one claims to have a soul, but I stick to my word that it is an entity that we don’t know yet.”
This argument won’t be resolved for some time. Moral issues surrounding the development of artificial intelligence demand attention now. The American pontiff has given his ally in Anthropic a platform for some hard conversations – if the AI overlords can stop their IPO campaigns long enough to engage in them.
This is the release Steven Levy Backchannel newsletter. Read previous newsletters Here.