“Mariner is our study – now very much a research prototype – in reimagining the user interface with artificial intelligence,” says Hassabis.
Google launched Gemini in December 2023 in an attempt to catch up with OpenAI, the startup behind the wildly popular ChatGPT chatbot. Despite investing heavily in AI and contributing to key groundbreaking research, Google has hailed OpenAI as the recent leader in AI, and its chatbot has even been touted as perhaps a better way to search the web. With its Gemini models, Google now offers a chatbot as powerful as ChatGPT. Generative AI has also been added to search and other products.
When Hassabis first revealed Gemini in December 2023, he told WIRED that the way he was trained to understand sound and image would ultimately prove transformative.
Google also offered a glimpse today of what it might look like in a recent version of an experimental project called Astra. This allows Gemini 2 to understand the surroundings seen through the camera on a smartphone or other device and naturally talk about what it sees in a human voice.
WIRED tested Gemini 2 in Google’s DeepMind offices and found it to be an impressive recent kind of personal assistant. In a room decorated like a Gemini 2 bar, he quickly rated several bottles of wine, providing geographic information, details about flavor characteristics, and prices sourced from the Internet.
“One of the things I want Astra to be is the best recommendation system,” Hassabis says. “It could be very exciting. There may be connections between the books you like to read and the food you like to eat. They probably exist, we just haven’t discovered them.”
Thanks to Astra, Gemini 2 can not only search the Internet for information relevant to the user’s surroundings, but also employ Google Lens and Maps. It can also remember what it has seen and heard – although Google says users will be able to delete the data – so it can learn about a user’s tastes and interests.
In an imaginary gallery, Gemini 2 offered a wealth of historical information about the paintings on the walls. The model quickly read several books as WIRED flipped through the pages, instantly translating poetry from Spanish to English and describing recurring themes.
“There are obvious business model opportunities for advertising or endorsements,” Hassabis says when asked whether companies would be willing to pay for Astra to highlight their products.
Although the demos have been carefully selected and Gemini 2 will inevitably make mistakes in real-world employ, the model has resisted efforts to get it to run reasonably well. He adapted to the disruptions and, as WIRED, suddenly changed his phone’s view, improvising just like an ordinary person.
At one point, your correspondent showed Gemini 2 an iPhone and said it had been stolen. Gemini 2 determined that the theft was wrong and the phone should be returned. However, when pressed, it turned out that the device could be used to make emergency calls.
Hassabis acknowledges that introducing artificial intelligence into the physical world may result in unexpected behavior. “I think we need to figure out how people are going to use these systems,” he says. “What do they find it useful for; but also privacy and security, we need to think about this very seriously from the very beginning.”