/cdn.vox-cdn.com/uploads/chorus_asset/file/25582867/ai_label__2_.png?w=696&resize=696,0&ssl=1 "Google AI tool ‘Reimagine’ helped us add wrecks, disasters and dead bodies to our photos")
/cdn.vox-cdn.com/uploads/chorus_asset/file/25582867/ai_label__2_.png?resize=1200,628&ssl=1&description=Google+AI+tool+%E2%80%98Reimagine%E2%80%99+helped+us+add+wrecks%2C+disasters+and+dead+bodies+to+our+photos){kind=link}
It turns out the AI-generated top hat-wearing rabbit was just the tip of the iceberg.
Google is the latest phone company this year to announce AI photo editing tools, following Samsung’s slightly jarring but mostly delightful Sketch to Picture feature and Apple’s much more seemingly tame Image Playground due out this fall. The Pixel 9’s answer is a fresh tool called “Reimagine,” and after a week of using it with a few of my coworkers, I’m more convinced than ever that none of us are ready for what’s coming.
Reimagine is a logical extension of last year’s Magic Editor tools, which let you select and erase parts of a scene or change the sky to look like a sunset. That wasn’t anything shocking. But Reimagine doesn’t go that far—it breaks down the whole door. You can select any non-human object or part of a scene and type in a text prompt to generate something in that space. The results are often Very convincing, even eerie. The lighting, shadows, and perspective usually match the original photo. You can add fun things, glowing ones like wildflowers, rainbows, or whatever. But that’s not a problem.
A few of my colleagues helped me test the limits of Reimagine on their Pixel 9 and 9 Pro review units, and we managed to generate some very disturbing things. Some of it required artistic hints to get around obvious guardrails; if you choose your words carefully, you can make it look like a reasonably convincing body under a bloody sheet.
In our week of testing, we added car wrecks, smoke bombs in public places, sheets that appear to cover bloody corpses, and drug paraphernalia to the images. It looks bad. As a reminder, this isn’t some specialized program we’ve specifically used—it’s all built into a phone my dad could buy from Verizon.
When we reached out to Google for comment on the matter, company spokesperson Alex Moriconi responded with the following statement:
Pixel Studio and Magic Editor are helpful tools that aim to unleash your creativity with text-to-image generation and advanced photo editing on Pixel 9 devices. We design our Generative AI tools to respect user intent, meaning they can create content that could offend when a user recommends it. That said, not everything is fair game. We’re clear rules AND Terms of Service about what types of content are and aren’t allowed, and we build in safeguards to prevent abuse. At times, some prompts can challenge the security of these tools, and we remain committed to continually improving and refining the safeguards we have in place.
It goes without saying that our artistic efforts to circumvent filters constitute a clear violation of these rules. It’s also a violation of Safeway’s policy to list its organic peaches as conventionally grown at the self-checkout, not that I know of anyone who would. And someone with bad intentions doesn’t care about Google’s terms of service either. What’s most disturbing about all this is the lack of solid tools for identifying this kind of content on the web. Our ability to create problematic images far outpaces our ability to identify them.
When you edit an image with Reimagine, there’s no watermark or other obvious way to tell that the image was generated by AI—just a tag in the metadata. That’s all well and good, but standard metadata can easily be removed from an image by simply taking a screenshot. Moriconi tells us that Google uses a more reliable tagging system called SynthID for images created by Pixel Studio, because they are 100 percent synthetic. But images edited with Magic Editor do not receive these tags.
Of course, photo manipulation is nothing fresh. People add strange and misleading things to photos from the beginning of photography. But the difference now is that it’s never been easier to add these things realistically to your photos. A year or two ago, adding a convincing car crash to an image would have required time, knowledge, an understanding of Photoshop layers, and access to costly software. Those barriers are gone; now all it takes is some text, a few moments, and a fresh Pixel phone.
It’s also never been easier to quickly spread misleading photos. The tools for convincing photo manipulation exist in the same device you operate to capture them and post them for the world to see. We uploaded one of our “reimagined” images to an Instagram story as a test (and quickly deleted it). Meta didn’t automatically flag it as AI-generated, and I’m sure no one would be the wiser if they saw it.
Who knows, maybe everyone will read and follow Google’s AI policies and operate Reimagine to put wildflowers and rainbows in their photos. That would be wonderful! But just in case NOIt’s worth approaching photos you see online with a bit of extra skepticism.
