Why it matters: OpenAI has a tool that could catch those people who use ChatGPT to write content while claiming it as their own. The text watermarking feature could be a revelation for teachers and companies who suspect people are cheating by using the chatbot but claim otherwise. However, OpenAI says it is only considering releasing it, possibly because doing so could impact its sales.
News that OpenAI has a system that can watermark ChatGPT-created text and detect it was revealed by The Wall Street Journal. The tool has been ready for about a year, but there is debate within the AI firm over whether it should ever be released.
In simple terms, the system works by making slight adjustments to the way ChatGPT selects words and phrases that follow each other, creating a pattern that can be detected by another tool – an invisible watermark, essentially.
OpenAI updated a May blog post following the publication of the WSJ report, confirming that "Our teams have developed a text watermarking method that we continue to consider as we research alternatives."
The post states that while the watermarking has proved highly accurate and effective against localized tampering, such as paraphrasing, it struggles when faced with globalized tampering, including using translation systems and rewording text using another generative model. Users could even circumvent the system by asking the model to insert a special character in between every word and then deleting that character.
OpenAI also claims that text watermarking has the potential to disproportionately impact some groups. It gives the example of stigmatizing the use of AI as a writing tool for non-native English speakers.
Text watermarking is one of several solutions OpenAI has been investigating. It has looked into classifiers and metadata as part of "extensive research on the area of text provenance."
In a statement to TechCrunch, an OpenAI spokesperson confirmed the WSJ's report but said it was taking a "deliberate approach" due to "the complexities involved and its likely impact on the broader ecosystem beyond OpenAI."
For all OpenAI's claims of why it is still considering releasing the watermarking method, the main reason is likely the fact that 30% of surveyed ChatGPT users said that they'd use the chatbot less often if watermarking was implemented.
In March 2023, a study by five computer scientists from the University of Maryland concluded that text generated by LLMs cannot be reliably detected in practical scenarios, both from a theoretical and practical standpoint. OpenAI seemed to agree, shutting down its AI classifier tool, which was supposed to determine the likelihood that a piece of text was written by an AI, in July last year due to its low rate of accuracy.