/cdn.vox-cdn.com/uploads/chorus_asset/file/25560987/Google_Home.jpg?w=696&resize=696,0&ssl=1 "Gemini Intelligence Comes to Google Home")
/cdn.vox-cdn.com/uploads/chorus_asset/file/25560987/Google_Home.jpg?resize=1200,628&ssl=1&description=Gemini+Intelligence+Comes+to+Google+Home){kind=link}
While flashy chatbots may get all the attention, generative AI has the real potential to make the astute home simpler and more accessible. Amazon has already announced its plans for a smarter Alexa to power your home. Now it’s Google’s turn to promise it can produce a better, smarter, more helpful Google Assistant.
Before the fall hardware event next week, Google announced three recent Gemini intelligence-powered experiences it plans to bring to the Google Home astute home platform later this year. There’s a recent camera intelligence feature that generates descriptive captions for Nest camera videos, natural-language input for creating Google Home routines, and a smarter Google Assistant for Nest astute speakers and displays with an all-new voice.
Most of these features — except for the recent voice — will be available for a fee as part of the Google service. Nest Aware Subscriptionits video recording subscription for Nest cameras, which starts at $8 per month ($80 per year). The features will launch first in Google’s Public Preview beta program for a narrow number of Nest Aware subscribers and will roll out to more users next year.
This is just the beginning of bringing more intelligence to the company’s astute home platform, said Anish Kattukaran, head of Google Home products. Edge in an interview before the announcement. “This sets the path for the next era of Google Home.”
All of this information will come as welcome news to long-suffering Google Home users, many of whom are tired of using unskilled, aging astute displays and watching the functions they rely on are canceled. They also struggled with the arduous transition from the Nest app to the Google Home app.
This week’s launch of the Google TV Streamer 4K (the hub of Google Home) and the recent Nest Learning Thermostat, as well as the promise of a smarter Google Assistant, mean things are starting to look promising for Google.
Google Assistant also looks like it’s here to stay. Instead of grafting Gemini onto Nest speakers and astute displays to control your astute home, Google is implementing Gemini intelligence behind the scenes. “Gemini is a family of models, and we’re optimizing it for Google Home components,” Kattukaran explains.
a:hover]:text-gray-63 [&>a:hover]:shadow-underline-black dark:[&>a:hover]:text-gray-bd dark:[&>a:hover]:shadow-underline-gray [&>a]:shadow-underline-gray-63 dark:[&>a]:text-gray-bd dark:[&>a]:shadow-underline-gray”>Graphics: Google Nest
Google is using Gemini intelligence in Nest cameras to let them understand what they see and hear, and then tell you what’s most vital. That means instead of just getting alerts about a person or package and then watching a video to see what happened, Google Home will add a detailed description of what the camera saw. The models will learn and train on your data — in the cloud, but for your home — getting smarter over time to better understand what’s happening around your home.
Kattukaran gave an example of a clip of a person unloading groceries from a car, with the caption:
A teenage person in casual clothing stands next to a parked black SUV. They are carrying shopping bags. The car is partially in the garage and the area appears tranquil.
Interpretive details aside, the caption provides a lot of context that, in addition to being helpful, can translate into smarter home automation. For example, if the camera detects an animal and understands that “dog is digging in the garden,” the next step might be to create an automation that “turns on the sprinklers.”
There will also be an option to search for text in videos in the Google Home activity tab. This could come in handy if, for example, my cat sneaks out after gloomy. I can ask it to show me the last time it saw the cat, rather than having to scroll through each video tagged with the animal to find it.
a:hover]:text-gray-63 [&>a:hover]:shadow-underline-black dark:[&>a:hover]:text-gray-bd dark:[&>a:hover]:shadow-underline-gray [&>a]:shadow-underline-gray-63 dark:[&>a]:text-gray-bd dark:[&>a]:shadow-underline-gray”>Image: Google Home
The recent “Help Me Create” feature in the Google Home app lets you describe a desired action — for example, “lock the doors and turn off the lights before going to bed” — and create a Routine that automatically performs that action.
You’ll need to operate text or speech input in the Home app on your phone (it doesn’t work through Nest speakers), but Kattukaran says it’ll have all the current capabilities of the Google Home app. That includes all the current starters, conditions and actionsand access to any device connected to Google Home, including Matter devices. It’s not as complicated or sophisticated as Google’s script editor, he says, but it should make it easier for anyone to create automations.
In addition to easier automation and camera intelligence, Google says it’s improving “core features” of its Google Assistant — like playing music and setting timers — on all current Nest astute speakers and displays.