I hate to admit it, but I spent a lot of money online during the holiday shopping season. And it’s no wonder that some of these purchases did not meet my expectations. A photo book I bought was damaged in transit, so I took some photos, emailed them to the seller, and got my money back. Online shopping platforms have long relied on photos submitted by customers to confirm that refund requests are legitimate. But generative AI is now starting to break this system.
A pinch too suspicious
On the Chinese social media app RedNote, WIRED found at least a dozen posts from e-commerce sellers and customer service representatives complaining about refund claims they received, allegedly generated by artificial intelligence. In one case, a customer complained that the sheet he purchased was torn to pieces, but the Chinese characters on the shipping label looked like gibberish. In another, a buyer sent a photo of a coffee mug with cracks that looked like paper tears. “It’s a ceramic mug, not a cardboard one. Who could tear a ceramic mug into layers like that?” the seller wrote.
Sellers reported that there are several product categories where AI-generated damage photos are most commonly used: fresh groceries, cheap cosmeticsand tender items such as ceramic mugs. Merchants often do not ask customers to return these items before issuing a refund, which increases their vulnerability to return fraud.
In November, a seller selling live crabs on Douyin, China’s version of TikTok, received a photo from a customer that appeared to show most of the crabs she purchased already arrived dead and two others had escaped. The buyer even uploaded videos showing dead crabs being poked with a human finger. But something was wrong.
“My family has been raising crabs for over 30 years. We have never seen a dead crab with its legs up,” said Gao Jing, a seller, in a video she later posted on Douyin. But what ultimately revealed the fraud was the sex of the crabs. The first video featured two men and four women, while the second clip featured three men and three women. One of them also had nine legs instead of eight.
According to a police complaint Gao shared online, Gao later reported the fraud to police, who determined the videos were indeed fabricated and detained the buyer for eight days. The case received widespread attention on Chinese social media, in part because it was the first known AI refund scam of its kind to trigger a regulatory response.
Lowering barriers
This problem is not exclusive to China. Forter, a Recent York-based fraud detection company, estimates that the number of AI-modified images used in refund claims has increased by more than 15 percent since the beginning of the year and continues to grow globally.
“This trend started in mid-2024 but has accelerated over the past year as image generation tools have become widely available and incredibly easy to use.” says Michael Reitblat, CEO and co-founder of Fortera. He adds that AI doesn’t have to get everything right because frontline retail workers and returns teams may not have time to thoroughly analyze every photo.