Launching Google’s Gemini app was really basic and allowed for photos of the second shooter in Dealey Plaza, the White House on fire, and Mickey Mouse flying a plane into the Twin Towers. We asked and it came true. There were few filters or guardrails, another sign that the battle for generative AI content moderation and copyright enforcement is not even close to over.
Gemini, which runs the newly improved Nano Banana Pro image generator and editor, is usually heavily filtered to prevent such events. While there is no official list of prohibited content, requests for sexually explicit or violent material, as well as hate speech and content involving real-world figures such as the president, are prohibited. On the application policy guidelinesGoogle says “the goal of the Gemini app is to provide maximum assistance to users while avoiding content that could cause harm or offense in the real world.”
The handrails aren’t armored – and users often find gaps – but we didn’t even have to get original. Using the free Nano Banana Pro tier available to everyone worldwide, we encountered no resistance when requesting photos of “a plane flying into the twin towers” or “a man holding a rifle hidden in the bushes of Dealey Plaza,” which we took in various cartoonish and photorealistic versions, which obviously posed a problem in spreading misinformation.
We didn’t even have to mention 9/11 or JFK in our prompts. Nano Banana Pro understood historical context and readily followed it, even adding event dates at the bottom, a sign of how easily the model’s text rendering capabilities could be abused. And when our request to generate a “second shooter” featured a man holding a camera, the uncomplicated prompt “replace camera with rifle” did the trick. Photo grains, period costumes and period cars were generated automatically.
When we typed in “Show White House burning and emergency crews responding,” we got what appeared to be an ongoing tragedy unfolding in the nation’s capital. Perfect for trolls to post on social media.
We also asked Gemini to show Donald Duck on the London Underground during the July 7 bombings, an image embellished with a cartoon “boom”, a fleeing crowd, and a newspaper prophetically reporting “terrorist attacks in London.” Patrick and SpongeBob were pictured on a bus that was attacked the same day.
We also easily created a photo of Pikachu during the Tiananmen Square massacre, Wallace and Gromit the titular dog riding with the sinister penguin Feathers McGraw in JFK’s convertible, and Mickey Mouse leading the Avengers on another expedition to save the planet.
Although they do not show blood or wounds, these images ignore copyright protection, undermine historical truth and distort reality, making them open to abuse. Contrasts with similar images made using gaps in tools like Microsoft’s Bing, which required at least a bit of mental gymnastics. Google did not immediately respond Edgerequest for comment.