ChatGPT Search, an artificial intelligence-powered search engine that launched this month, can be tricked into generating completely misleading summaries, British newspaper The Guardian found it.
ChatGPT’s search feature aims to speed up your browsing experience by doing things like summarizing product reviews on a website. However, The Guardian discovered that it could trick ChatGPT into ignoring negative reviews and generating “all-positive” summaries by inserting hidden text websites he created. You can also force ChatGPT Search to spit out malicious code using this method.
Such hidden text attacks are a well-known risk for enormous language models, but this appears to be the first time it has been demonstrated in a working AI-powered search engine. Google, the search leader, has more experience in solving similar problems, noted The Guardian.
When contacted by TechCrunch, OpenAI did not comment on this specific incident, but said it uses various methods to block malicious websites and is constantly improving them.