If you want to make the best employ of the world, more and more filled with AI tools, here is the habit for development: start making screenshots. Many screenshots. Everything and everything. Because in the case of all voice modes, ubiquitous cameras and the multimodal future of everything, there can be no more valuable digital behavior than pressing the buttons and saving what you are looking at.
Screenshots are the most universal method of capturing digital information. You can capture everything – well, almost everything Thanks a lot, Netflix! – With a few clicks and save and share it with almost any device, application or person. “This is the portable data format,” says Johnny Bree, the founder of the Digital Storage application Fabric. “There is nothing that is so portable that you can move between any software.”
The screenshot contains a lot of information, such as its source, content and even time of the day in the corner of the screen. First of all, it sends a key and elaborate signal; says I take care of it. We have countless up-to-date AI tools that are aimed at watching the world, our lives and everything, and trying to all for us. These tools are mostly nonsense for many reasons, but mainly because AI is quite good in the knowledge of what it is, but the garbage in whether they matter. The screenshot assigns a value and tells the system that he must pay attention to.
Screenshots are also placed by the user in an crucial way of control. “If I give you access to all my e -Maili, all my WhatsApps, everything, there is a lot of noise,” says Mattias Deserti, head of smartphone marketing for nothing. There is simply no reason to save any e -mail or every website you visit – and this says nothing about the consequences of privacy. “What if you were able to start training the system yourself instead, supplying the information system to want A system that you should know about you? “Instead of a tool such as Microsoft Recare, which asks for unlimited access to everything, starting with screenshots, allows you to choose what you share.
Until now, screenshots were a fairly blunt instrument. One snow, and it is saved to the camera roll, where there is probably no, forgotten until the end of the time. (And don’t start me on all screenshots that I take by accident, mainly from my locksmith screen.) At best, you can search for text in the picture. But it is more likely that you will have to predict until you find it again.
The first step in making screenshots is more useful to determine what exactly is in them
The first step in making screenshots is more useful, it is to determine what exactly is in them. It is at the beginning it differs terribly complicated: the technology of recognizing optical characters has long done a good job, detecting the text on the page. Models of artificial intelligence go a step further, so you can view the title or simply “movies” to find all digital snaps of posters, fandango results, Tiktok recommendations and many others. “We use the OCR model,” says Shenaz Zack, product manager at Google and part of the band behind the Pixel Screenshots application. “Then we use the entity detection model, and then the twins to understand the actual context of the screen.”
You see, the screenshot has much more than just the text inside. The right AI model should be able to say that it comes from WhatsApp, only according to a specific green color. He should be able to identify a website based on the header logo or understand when you write the name of Spotify, a review of the Yelp Handyman or Amazon list. Armed with this information, the application of the screenshot can start to automatically organize all these images for you. And even this is just the beginning.
With everything I have described so far, everything we have really created is a very good application to look at your screenshots, which really thinks it is a good idea, because it would be one more thing to check – or forget to check. Where it becomes much more captivating when the device or application can start using the screens on your behalf to lend a hand you remember what you intercepted and even employ this information to do things.
For example, in the up-to-date space application nothing, the application can generate reminders based on saved items. If you take a screenshot of the concert you want to go to, it may remind you that it will appear automatically. Pixel Screenshots push this idea even more: if you save a list of concerts, your Pixel phone may lead to listen to this band next time you open Spotify. If you drop a screenshot, identification card or on -board pass, you may ask you to place it in the portfolio application. The idea, as Zack says, is about thinking about screenshots as an input system for everything else.
Mike Choi, an independent programmer, built an application called Camp Partly to lend a hand him employ his own screenshots. He began working on transforming each screenshot into a “card”, with crucial information stored next to the photo. “You have a screenshot and there is a button downstairs and turns the card,” he says. “It shows a map if it was a location; a song preview, if it is a song. The idea was, given the infinite pool of different types of screenshots, or AI can simply generate the perfect user interface for this category in flight?”
If all this sounds familiar, this is because there is a different date for what is happening here: it’s called Agentic AI. Each company in technology seems to work on how to employ artificial intelligence to achieve things on your behalf. In this case, you just don’t have to write long hints or conversations with the assistant. You just take a screenshot and let the system go to work. “You are building a knowledge base when this knowledge base today is limited to your gallery and nothing happens to it,” says Deserti. He is excited to reach the point where you drop the date of the concert, and the necessary space automatically encourages the purchase of tickets during sale.
Reservation of screenshots is not always so plain
However, understanding of screenshots is not always so plain. Some want to stop forever, as often you need an identification card; Other things, such as a concert poster or parking pass, have a very constrained life on the shelf. As for how the application is to distinguish between the parking pass, which you employ every day at work, and the one you used once at the airport and you never need again? Some screenshots on my phone were sent to me on WhatsApp; Others that I caught from instagram memes to send to friends. No one should be fully kept against them, and the same applies to screenshots. Many of these screenshot applications are looking for ways to quickly add a note or organize things yourself to provide additional helpful information in the system. But it is challenging to do it without ruining what makes the screenshots so velvety and basic.
One of the ways to solve this problem, make screenshots even more automatically useful, is to collect an additional context from the device. At this point, companies such as Google and nothing has the advantage: because they create a device, they see everything that happens when you do a screenshot. If you beat a screenshot from your web browser, they can also store the link you looked at. They can also see your physical location or record time and weather. Sometimes it is all useful, but sometimes it is nonsense; The more data they collect, the more these applications risk encountering the same buzzing problem, which screenshots helped solve first.
But the input system works. We all take screenshots all the time, and we are used to taking them as a way to place a tag so many types of useful information. Getting access to this type of crucial, personalized data is the most complex to build a great AI assistant. The future of calculations is certainly multimodal, including cameras, microphones and sensors of all kinds. But the first best way to employ AI may be one screenshot at once.