Microsoft uses artificial intelligence to improve search in Windows

Microsoft is using artificial intelligence models to significantly improve Windows search on its recent Copilot Plus computers, including adding a recent click-to-do feature very similar to Google’s Circle to Search feature. These search improvements will make it easier to find and interact with images, emails, documents, and even videos. These are just some of the AI-powered features that will be available on Copilot Plus computers from November.

An improved Windows search engine will appear for the first time in File Explorer on Copilot Plus computers next month, allowing you to search for images using words even if the search word is not found in the photo or file name.

“AI-powered search makes it much easier to find virtually anything,” says Yusuf Mehdi, executive vice president and chief consumer marketing officer at Microsoft. “You no longer need to remember file names and document locations, or even specific word names. Windows will better understand your intentions and match the appropriate document, image, file or email.

This improved search will also be available “in the coming months” in the main Windows search interface and in the search box displayed in the Settings interface. In the settings search box, you can type something like “add my headphones” to help you find the right settings.

Windows search hasn’t been very good for years, so this AI-powered natural language search should improve things significantly – provided it works as well as Microsoft promises. Microsoft is using NPU chips in its new Copilot Plus computers to enable local searches for OneDrive content without the need to connect to the Internet.

In addition to Windows Search, Microsoft will next month begin rolling out Click to Do on Copilot Plus computers, which is very similar to Google’s Circle to Search feature. With Click to Do, you press the Windows key on your keyboard and left-click to display an interactive overlay on your screen that allows you to select images or text to perform clickable actions.

“‘Click to Do’ is about understanding everything you see on your screen and incorporating useful shortcuts into actions that help you search, learn, edit, shop or react to it faster,” explains Mehdi. “It works on any window, document, image, or even video.”

You can apply the Click to Do feature on a YouTube video you’re watching to perform a visual search for an item displayed in the video using Bing. Click to Do is also context-sensitive, so it can support with text-related activities such as rewriting or summarizing documents, explaining text, and sending emails.

Microsoft will begin testing the recent search and click-to-do features with Windows Insiders on Copilot Plus computers in October, with a phased rollout in November. The previously announced Recall feature will also be available to testers in October on Qualcomm-based devices, and then in November it will be available to Windows testers on Copilot Plus computers with Intel or AMD processors. Microsoft says “details on Recall’s wide availability schedule will be made available soon.”