Often in the evenings, and sometimes on weekends, when the robots weren’t busy with their daily chores, Catie and her improvised team would gather a dozen or so robots in a enormous atrium in the middle of the X. The swarms of robots began to move together, sometimes hesitantly but always in captivating patterns, with what often seemed curiosity, sometimes even grace and beauty. Tom Engbersen is a roboticist from the Netherlands who painted replicas of classical masterpieces in his spare time. He started a side project with Catie, exploring how dancing robots could respond to music, or even play an instrument. At some point, he had a novel idea: What if the robots themselves became instruments? This began an exploration in which each joint of the robot made a sound as it moved. When the base moved, it made a bass sound; when the gripper opened and closed, it made a bell sound. When we turned on the music mode, the robots created unique orchestral scores every time they moved. Whether they were moving down a hallway, sorting trash, clearing tables, or “dancing” in a herd, the robots moved and made sounds like a recent breed of approachable creature, unlike any I had ever encountered.
This is just the beginning
In overdue 2022, the end-to-end versus hybrid conversations were still ongoing. Peter and his teammates, along with our colleagues at Google Brain, were working to apply reinforcement learning, imitation learning, and transformers—the architecture behind LLM—to several robotic tasks. They were making good progress in showing that robots could learn tasks in a way that made them general, tough, and resilient. Meanwhile, the applications team led by Benjie was working to exploit AI models and conventional programming to prototype and build robotic services that could be deployed to humans in real-world settings.
Meanwhile, Project Starling, as Catie’s multi-robot installation was eventually called, was changing my feelings about these machines. I noticed how people were drawn to the robots with awe, joy, and curiosity. It helped me understand that How robots move among us, and the sounds they make evoke deep human emotions. They will have a gigantic impact on how (if at all) we accept them in our daily lives.
In other words, we were on the verge of truly exploiting the biggest bet we had ever made: AI-powered robots. AI gave them the ability understand what they heard (spoken and written language) and translate it into actions or understand what they saw (camera images) and translate it into scenes and objects they could interact with. And as Peter’s team showed, the robots had scholar to pick up objects. More than seven years later, we’ve deployed fleets of robots to multiple Google buildings. One type of robot has performed a range of services: autonomously wiping down tables in coffee shops, checking conference rooms, sorting trash, and more.
That was when, in January 2023, two months after OpenAI launched ChatGPT, Google shut down Everyday Robots, citing general cost concerns. The robots and a tiny number of people eventually landed at Google DeepMind to conduct research. Despite the high cost and long timeline, everyone involved was shocked.
National Order
In 1970, there were 10 people of working age for every person over 64 in the world. By 2050, there will likely be fewer than four. We are running out of workers. Who will care for the elderly? Who will work in factories, hospitals, restaurants? Who will drive trucks and taxis? Countries like Japan, China, and South Korea understand the urgency of this problem. There, robots are not optional. These countries have made investing in robotic technology a national imperative.
Giving AI a real-world presence is both a national security issue and a huge economic opportunity. If a tech company like Google decides it can’t invest in “bull’s eye” projects like AI-powered robots that will supplement and complement the workers of the future, who will? Will Silicon Valley or other startup ecosystems rise to the occasion, and if so, will there be patient, long-term capital available? I have my doubts. The reason we called Everyday Robots a bull’s eye is that building highly elaborate systems at this scale far exceeded what venture-backed startups had the patience for. While the United States is at the forefront of AI, building its physical manifestation—robots—requires skills and infrastructure that other nations, notably China, already excel at.
The robots didn’t arrive in time to assist my mother. She died in early 2021. Our recurrent conversations toward the end of her life convinced me more than ever that a future version of what we started at Everyday Robots was coming. In fact, it couldn’t come soon enough. So the question we have to ponder is: How does this kind of change and future happen? I remain curious and concerned.
Let us know what you think about this article. Send a letter to the editor at Email address: mail@wired.com.