Photo: Stephanie Arnett/MIT Technology Review | Adobe Stock
On October 23, Google DeepMind announced that it had made its AI watermarking tool, SynthID, open source.
The SynthID tool, already applied in Google’s Gemini chatbot, aims to address concerns over AI-generated content by embedding a watermarking into the text.
While it has received mixed responses regarding its effectiveness, the watermarking approach uses an invisible “statistical signature” embedded into the text as it’s generated.
This digital identifier makes it easier for companies, educators, and governments to determine if any content originated from AI.
The release comes after a growing push for AI transparency and accountability, especially in the face of misinformation, academic cheating and plagiarism.
“While no known watermarking method is foolproof, I really think this can help in catching some fraction of AI-generated misinformation, academic cheating and more,” says Scott Aaronson at The University of Texas at Austin, who previously worked on AI safety at OpenAI.
How will it work?
SynthID works by modifying the probability of word selection during the generation process.
Tokens—parts of words, characters, or entire phrases—are subtly adjusted so that the generated text carries an invisible watermark.
These adjustments do not affect the content's readability or meaning for users but embed a signature detectable by specialised software.
Google’s watermarking uses a technique called “tournament sampling,” where different word options are tested against each other, creating watermarked text that can later be verified as AI-generated.
In an initial live test with 20 million Gemini chatbot users, scientist Pushmeet Kohli and his team found that “users didn't notice any difference in quality between watermarked and non-watermarked content.”
“Now, other [generative] AI developers will be able to use this technology to help them detect whether text outputs have come from their own [large language models], making it easier for more developers to build AI responsibly,” says Kohli, the vice president of research at Google DeepMind.
The watermark is designed to be imperceptible so users don't perceive it as altering the natural flow of language.
However, "achieving reliable and imperceptible watermarking of AI-generated text is fundamentally challenging," notes Soheil Feizi, an associate professor at the University of Maryland.
The problem is that AI models generating deterministic outputs, like factual answers or code, present more difficulties when trying to embed a watermark without affecting accuracy.
The open-source release of SynthID invites developers to further explore and enhance its reliability.
While Google DeepMind believes that broader adoption will strengthen SynthID's capabilities, concerns remain over its vulnerability to tampering.
"It is still comparatively easy for a determined individual to remove a watermark," warns a study published in Nature, underlining that paraphrasing, translating, or using another AI tool can quickly erase the watermark.
The path ahead
Google DeepMind's decision to make SynthID open-source aims to address the ongoing challenge of AI-generated misinformation.
Experts like Irene Solaiman, head of global policy at Hugging Face, stress that watermarking alone is insufficient to address AI misuse.
“Watermarking is one aspect of safer models in an ecosystem that needs many complementing safeguards,” she said, pointing out the need for a multi-layered approach that includes regulation, technological improvements, and public awareness.
With growing concerns around misinformation and misuse, the urgency for comprehensive solutions like SynthID is becoming increasingly clear.
Several regulatory bodies are pushing ahead with requirements to make watermarking AI-generated content mandatory.
The European Union’s AI Act already imposes strict regulations, while China has moved ahead with mandatory watermarking requirements.
In the United States, the National Institute of Standards and Technology (NIST) is developing safety standards in response to President Joe Biden's directive in October 2024.
SynthID may be a promising step towards tackling some of the issues surrounding AI-generated content, but as many experts have pointed out, it is only one piece of a larger puzzle.
“With better accessibility and the ability to confirm its capabilities, I want to believe that watermarking will become the standard,” said Joao Gante, a machine-learning engineer at Hugging Face.
While SynthID shows promise, it is clear that it must still overcome significant technical and adoption hurdles before becoming a reliable standard for identifying AI-generated content.
Copyright © 2024 TRT World.