OpenAI won’t watermark ChatGPT text because its users could get caught

OpenAI has had a text watermarking system created by ChatGPT and a watermark detection tool ready for about a year, reports Wall Street Journal. But the company is internally divided over whether to release it. On one hand, it seems like the responsible thing to do; on the other, it could hurt its bottom line.

OpenAI watermarking involves adjusting how a model predicts the most likely words and phrases to appear after the previous ones, creating a detectable pattern. (This is an oversimplification, but to learn more, check out Google’s more detailed explanation of text watermarking in Gemini .)

Offering any way to spot material written by AI is a potential boon for teachers trying to stop students from handing in written work to AI. Journal reports that the company found that watermarking does not affect the quality of its chatbot’s output. In a study commissioned by the company, “people around the world supported the idea of ​​an AI detection tool by a margin of four to one,” Journal I’m writing.

After Journal published its paper, OpenAI confirmed that it was working on text watermarking in today’s blog post update it was noticed by TechCrunch. The company claims that its method is very exact (“99.9% effective”, according to the documents. Journal saw) and resistant to “manipulations like paraphrasing.” However, it says techniques like rephrasing with a different model make them “easy for bad actors to bypass.” The company also says it is concerned about the AI ​​tools’ usefulness in stigmatizing non-native speakers.

However, OpenAI also appears to be concerned that the exploit of watermarks could discourage ChatGPT users surveyed, with nearly 30 percent of whom told the company they would be less likely to exploit the software if watermarks were introduced.

Despite this, some workers still claim that watermarking is effective. However, in lithe of the persistent feelings of users, Journal says some have suggested trying methods that are “potentially less controversial among users, but unproven.” In a blog post update today, the company said it is “in the early stages” of exploring metadata embedding. It says it’s still “too early” to know how well it will work, but because it’s cryptographically signed, there won’t be any false positives.