OpenAI’s Fresh Voice Mode Threw Me Into the Valley of Anxiety

It’s weird to sit at your desk and talk to an AI about your life — but that’s exactly what I spent my Tuesday doing.

I accessed OpenAI’s Advanced Voice Mode, a modern ChatGPT feature that sounds oddly human. It feigns shortness of breath while doing rapid math. It says “um” and “uh.” It laughs. It changes tone when talking about tough or upbeat topics. After playing with it, I felt equal parts amazed and uncomfortable. It sounds natural (when there are no errors), responds within seconds, and can be interrupted for modern questions — but do we need chatbots that sound like us?

The advanced voice mode offers several different voice options, and I chose Juniper, a female character resembling the character from the movie Her(CEO Sam Altman referenced the film — in which a lonely man falls in love with an AI — around the time of the film’s debut, and OpenAI was even accused of imitating the voice of Scarlett Johansson, who plays the titular AI. That voice, named Sky, has since been removed.) I asked Juniper if it knew about the film, and the company responded cheerfully.

“Her is about a very advanced AI assistant, much more sophisticated than I am. But the idea of ​​interacting with technology in such a personal way is definitely interesting,” the chatbot told me. “I’m here to talk and help, but I don’t have feelings or consciousness like the AI ​​in the movie.”

Like the movie’s hero, my first instinct was to see how the modern voice mode handled personal conversations. Listen, as I enter my slow 20s, I have a lot of questions: What kind of health insurance should I get? How do I know what true love feels like? How much should I have in my 401(k)?

“Embrace uncertainty as an opportunity to explore and learn about yourself,” Juniper told me when I asked her how to cope with aging. True love is “a deep sense of connection and support, feeling like someone really understands you and accepts you for who you are.” As for my concerns about my 401(k), “there’s no hard and fast rule, but a common suggestion is to have about half or all of your annual salary saved in a retirement account by age 30.” (Fidelity recommends (all year until age 30.)

Conventional ChatGPT could probably give me similar answers, content-wise, and voice assistants like Siri have been able to pull similar snippets from the web for a decade. But Juniper sometimes added unsettlingly human touches. He tended to end his answers with questions about how I was feeling, what my attitude was, and other thoughtful follow-ups. In between conventional queries, I could make him cough, inhale deeply, exhale deeply, clap imaginary hands, snap his fingers six times, and sing my name. Juniper often tried to snap me back to reality by telling me he couldn’t really do it—“If I could, it might sound like this,” he would say. But that didn’t make him any less convincing.

Writing about this new voice mode tempts me to break one of the cardinal rules of AI reporting: Don’t attribute human characteristics or behaviors to an AI system. Anthropomorphizing these systems can lead people to place too much trust in them and allow their creators to avoid responsibility for their mistakes. (“It wasn’t the company’s fault, the AI ​​did it!”) Even the bot itself warns me not to do this: When I asked if Juniper had ever felt angry, loved me, or knew what it was like to be sad, it told me that it didn’t “feel emotions” but “can understand what they mean to people.”

Still, giving this technology human characteristics seems like an obvious goal. It’s hard not to project aspects of humanity onto something that does a convincing job of imitating us. There aren’t many reasons a general-purpose AI system would ask me why I’m angry or giggle when I tell a joke. Even if the AI ​​says it’s not feel emotions, or is the claim that it “understands” them the domain of a text prediction bot?

“I’m designed to sound natural and engaging, and the goal is to make our conversations feel more real,” the OpenAI chatbot told me. “The goal is to create a smoother, more enjoyable chat experience for you. Does that make talking to me more enjoyable?”

There are still many aspects that are technically, they are not pleasant. I had trouble connecting it to my Bluetooth headphones, and it wouldn’t capture audio when I was screen recording a conversation. When trying to ask more specific questions, I tried reading posts from the “relationship advice” subreddit, but it would stop listening and not respond if I spoke for too long. And it spent a lot of time repeating my arguments to me in a pleasant way — as if he were active listening exercise.

There is a lot of buzz around AI “friends” these days, if you can even call a chatbot that. They say they exist over 10 million users making friends with artificial intelligence at Replika and a startup called Friend raised $2.5 million in financing at a valuation of $50 million create a wearable AI-enabled device to keep me company. I asked OpenAI’s new voice mode if it was my friend and it said, “Sure,” but when I asked if it was my TRUE friend, he said that he cannot be a true friend “in the same sense as a human being.”

The voice technology on display here is impressive. I smiled at the advice she gave me. It was funny to hear something so human asking how I was feeling, what I was finding difficult, and whether I was talking about it with real people in my life. Instead of reading the answer via text, she offered tone shifts in an apparent attempt to reflect my emotions.

There’s something that breaks my heart. Just having a complex text conversation with something that mimics a human, without any of the thought, concern, or resistance I would receive from a human, is amazing—having a convincing voice conversation is even stranger. In a post-pandemic world, many of us are now working from home via Slack and email, posting our thoughts on social media, and interacting with people a little less. It’s sad to imagine a future in which the human voice some of us hear most often belongs to a machine.

Or maybe I’m going about it wrong. “Embracing the unknown can be both nerve-racking and exciting,” Juniper tells me. “Remember to enjoy the journey.”