Apple Visual Intelligence is a built-in version of Google Lens

Apple has announced a novel feature called Visual Intelligence that will become part of iOS 18’s Apple Intelligence AI suite “later this year.” The feature works similarly to similar features offered by other multi-modal AI systems from Google or OpenAI.

Visual Intelligence lets you “instantly learn about everything you see,” Apple’s Craig Federighi said during the company’s September event today. Federighi said the feature is “enabled by Camera Control,” which is the company’s name for the novel capacitive camera button that’s now on the side of the iPhone 16 and 16 Pro. To trigger it, users will have to tap and hold the button, then point the phone’s camera at what interests them.

iPhones apply “a combination of on-device intelligence and Apple services that never store your photos” to power Visual Intelligence, letting you snap a photo of a restaurant to get its hours of operation. Point the camera at a flyer and “details like the title, date, and location are automatically captured,” he said. Federighi added that the feature is “also your gateway to third-party models,” suggesting using Visual Intelligence to search Google for a bike you found in the wild or take a photo of your college notes to get support with a concept.

Apple did not provide a release date for this feature, only saying that it “will be available in the Camera Control app later this year.”