Google Lens now allows you to search using video

If you can’t capture what you’re looking for with just a photo, Google Lens will now let you record a video and even ask what you’re seeing using your voice. This feature will display an AI overview and search results based on the video content and your question. This feature is now available in Search Labs on Android and iOS devices.

Google first released video previews for I/O searches in May. For example, Google says that a person curious about the fish they see in an aquarium can bring their phone close to the exhibit, open the Google Lens app, and then hold the shutter button. Once Lens starts recording, it can ask, “Why are they swimming together?” Google Lens then uses the Gemini AI model to provide a response similar to what you see in the GIF below.

There’s no way yet to recognize sounds in a video – for example, when trying to identify a bird you hear – but Patel says Google is “experimenting with this.”

Google Lens is also updating its photo search function with the ability to ask questions by voice. To try it out, point your camera at your subject, hold the shutter button, and then ask a question. Before this change, you could only enter a question in Lens after taking a photo. Voice Questions is available globally on Android and iOS devices, but is only available in English for now.