OpenAI text watermark detection drops to 17% after 25% of words change

OpenAI yesterday said it will add an invisible watermark to eligible ChatGPT and Codex text in the European Union over the coming weeks, open an opt-in version to API customers worldwide, and restrict its detector to approved researchers and expert organizations. The company framed the step as a response to the EU AI Act and published test results showing that the signal weakens with short passages and with editing.

In Short

OpenAI is starting to hide a statistical pattern in the words that ChatGPT and Codex write for people in the EU, so that a special detector can later say the text probably came from an OpenAI model. This matters because EU law expects AI companies to make generated text identifiable by machines, yet the pattern is hard to spot in short texts and mostly disappears when the text is reworded or translated. For now, API customers anywhere can switch the pattern on, ChatGPT and Codex users in the EU get it within weeks, and only approved researchers can check for it.

What OpenAI is rolling out

According to OpenAI, the plan has three strands, each on a different clock. Text from ChatGPT and Codex will carry an invisible watermark for eligible users on all plans in the EU, with the rollout spread over the coming weeks. Outside the bloc nothing changes by default: OpenAI wrote that it is "not making text watermarking a global default at launch", and described the regional approach as giving it "room to learn from real-world use and feedback."

API: opt-in, worldwide

API customers in any country can switch the feature on from today for select models, and it stays off unless a customer enables it. OpenAI said this lets customers "decide how watermarking fits their transparency obligations" as well as the experiences they offer to users. The documents name neither the models nor the criteria that make ChatGPT or Codex output eligible. OpenAI is also working with cloud partners on watermarking for OpenAI model outputs reached through their services, again within the coming weeks.

Detector: applications only

The detector is a separate matter. Approved researchers and expert organizations can apply for access starting today, and access will be granted case by case in the first phase. The tool reports only whether it finds an OpenAI watermark, without identifying the user or revealing prompts or conversations. Citing "the risk of missed watermarks and false positives", OpenAI is keeping it off the public web at launch.

Scope and what comes next

The measures cover text only. Verification tools for images and audio, including a web tool at openai.com/verify and a Content Provenance API, remain publicly accessible. OpenAI's image and audio tooling already pairs signed metadata under the C2PA standard with invisible watermarks, and the company began applying SynthID alongside C2PA to images from ChatGPT, Codex and its API in May 2026. Text needed its own treatment, the company's page says, because of "how easily it can be rewritten, translated, or edited."

OpenAI also plans to release the technology in open source, and said its technical report will be "updated with additional details in the coming weeks."

How the signal works, and where it fails

The technology, which OpenAI calls textGrain, adds an invisible statistical signal to the model's word choices as text is generated; the detector then tests a passage for that signal. According to OpenAI, no visible marks or special characters are added. In the company's own comparisons textGrain "matched or exceeded the performance of other approaches we tested", a group that included SynthID for text. The documents give no figures for those comparisons. Strong performance under ideal conditions, in OpenAI's phrasing, "does not guarantee reliable detection in everyday use".

Two kinds of error are possible. A detector can report a watermark where none exists (a false positive) or miss one that is present (a false negative). The detection figures below are reported at a target false positive rate of 1%.

Length and subject matter

Passage length drives the result. For psychology content, the detector found the watermark in about 80% of passages of 200 tokens and about 95% of passages of 400 tokens - roughly 150 and 300 words, on the usual rule of thumb of three-quarters of a word per token. Mathematics fared worse. OpenAI said detection rates were "substantially lower" for content with less flexibility in word choice, though the text of the page gives no percentages for it; a chart plots both subjects at 200, 300 and 400 tokens. OpenAI does not set out the mechanism, but the pattern fits a statistical test that gathers evidence across a passage, which would explain why short or tightly constrained text leaves less to find. In a post on X, the company put the point more bluntly: watermarks are "often undetectable, especially in short passages."

Editing and translation

Edits do more damage. In an evaluation of 400-token passages, replacing 10% of words with synonyms cut detection from about 92% to 66%. Replacing 25% of words left the watermark found in only 17% of passages - roughly one in six. The figures rest on English-language responses to questions from the ELI5 dataset. On X, OpenAI added: "Rewriting or translating text can completely remove the watermark."

Output quality: eight benchmarks

OpenAI also tested whether the signal alters model output. It compared watermarked and unwatermarked text from Astra, which it describes as its latest frontier model, across eight benchmarks. The company's page labels both columns "Astra, max". The differences below are calculated from OpenAI's published scores.

Five of the eight scores are higher with the watermark and three are lower. The widest gaps are 3.10 points on Terminal-Bench Science 0.1, in the watermarked column's favor, and 1.12 points on DeepSWE v1.1, against it. OpenAI concludes that it sees no meaningful performance differences with and without watermarking, and said in its X thread that testing showed no effect on speed or on how responses read. These are vendor-supplied figures. The page gives no run counts or confidence intervals, so the direction of such small gaps cannot be separated from run-to-run variation using the documents alone.

What a detection result does not establish

What, then, does a positive result show? Only the answer to one question, which OpenAI phrased as "Was this likely generated by an OpenAI model?" The company lists the limits of that answer:

• Human contribution: a watermark can indicate that an OpenAI system generated or processed part of a passage, but not how much human judgment, editing or creativity went into it.
• Ownership and responsibility: it does not determine who owns the text, whether its use was lawful, whether disclosure was required, or who is responsible for it.
• User identity: it does not associate a person, organization, account, prompt or conversation with the text.
• Accuracy: it says nothing about whether a passage is true, misleading, harmful or presented in the right context.
• Absence: in OpenAI's words, "The absence of a detected watermark does not prove human authorship." Text can be too short, edited or translated for detection to work, can come from an unsupported model, can predate watermarking, or can have been generated by another company's tools.

The regulatory backdrop

OpenAI said the EU AI Act requires generative AI providers "to make generated text identifiable in a machine-readable way", while describing text watermarking and detection as "early technologies with significant limitations".

Article 50 transparency obligations became applicable on August 2, 2026, after the European Commission published its guidelines and a finalised Code of Practice on Transparency of AI-Generated Content on July 20, 2026. The Code sets a threshold for text: free-form text longer than 200 tokens has to be watermarked, even though the Code concedes the marking may be less reliable than on longer passages. Signing is voluntary, and Section 1 also calls for providers to offer a free detection solution under a zero-retention rule. Non-compliance with Article 50 carries penalties of EUR 15,000,000 or 3 percent of worldwide annual turnover, whichever is higher. Earlier drafts of the Code had allowed a provenance certificate as an alternative to text watermarking; the final text dropped that option.

Other large providers have taken different routes. Google signed the Code on July 24, 2026 and Meta followed on July 28. Anthropic updated a help article on August 10, 2026 stating that marking applies wherever Claude is offered rather than only in the EU; detection tooling had not shipped at that point and older models remained unmarked. OpenAI's design differs on each count: a regional default for its consumer products, opt-in for the API, and a detector restricted to vetted applicants.

OpenAI's materials do not say whether the company has signed the Code. They do tie the case-by-case detector access to a Code of Practice that the page references through a link; the copy of the page supplied for this article does not display the link's target, so the document cannot settle which code is meant.

Two further dates frame the rollout. Systems already on the market before August 2, 2026 have until December 2, 2026 to bring machine-readable marking into conformity, and until February 2, 2027 to implement watermark-detection interoperability. OpenAI's materials mention neither date, and do not say whether the "coming weeks" schedule is timed to the first.

Why this matters to the marketing community

Most advertising text is short. When Anthropic published its own marking plans, PPC Land noted that a headline, a meta description, an ad callout or a product feed attribute may fall below the length at which a statistical text watermark can be recovered reliably. OpenAI's numbers point the same way: for the psychology content it tested, a 200-token passage is caught roughly four times in five, and the mathematics results were lower still.

A watermark is also not a label. The IAB Austria guide reads machine-readable signals such as SynthID and C2PA metadata as falling short of the visible disclosure that qualifying creative requires. The same split appears in coverage of Google's Lyria 3.5: an imperceptible watermark meets the provider-side marking duty without, by itself, meeting the deployer-side labelling duty. Under Article 50(4), deployers must disclose deep fakes and AI-generated text published to inform the public on matters of public interest, absent human editorial control.

Coverage will be uneven. ChatGPT and Codex text carries the mark only in the EU, API output carries it only when a customer opts in, and Anthropic applies marking wherever Claude is offered. Copy that passes between tools, agencies and languages therefore carries different signals depending on where it started, and OpenAI states that translation can remove its signal entirely. For multi-market campaigns, translation is a routine production step.

The opt-in design also hands a decision to companies building on the API. PPC Land reported that an advertising technology company building a creative generation product, a copy assistant or a campaign planning agent on a third-party foundation model is a downstream provider in the language of the AI Act. OpenAI leaves the choice of whether to switch watermarking on with those customers.

Verification is the last gap. The Code's free-detection requirement contrasts with a detector open only to vetted applicants at launch, and the interoperability deadline of February 2, 2027 is the next fixed date for detection. OpenAI offers no timetable beyond saying it will "revisit each part of this approach as the technology, standards, and evidence evolve."

Open questions

• Which API models count as "select", and what makes ChatGPT or Codex output eligible; the documents name neither.
• When cloud partners, the open-source release and the updated technical report will arrive, beyond "the coming weeks".
• What criteria approve a researcher or expert organization for detector access.
• Whether OpenAI has signed the EU Code of Practice, and how the closed detector fits its free-detection requirement.

Timeline

• July 20, 2026 - European Commission publishes Article 50 guidelines and the finalised Code of Practice on Transparency of AI-Generated Content.
• July 24, 2026 - Google says it is signing the Code of Practice.
• July 28, 2026 - Meta says it is signing the Code of Practice.
• August 2, 2026 - EU AI Act Article 50 transparency obligations become applicable.
• August 10, 2026 - Anthropic updates a help article on marking text and files generated by Claude models.
• October 5, 2026 (today) - OpenAI publishes its approach to EU text provenance rules; API opt-in watermarking and detector applications open.
• Over the coming weeks - Planned rollout to eligible ChatGPT and Codex users in the EU; cloud partner availability and technical report update also due.
• December 2, 2026 - Deadline for systems on the market before August 2, 2026 to bring machine-readable marking into conformity.
• February 2, 2027 - Deadline for providers to implement watermark-detection interoperability.

Related PPC Land coverage

• Claude text gains invisible watermarks across 5 Anthropic products - Anthropic's August 2026 marking plan, which applies wherever Claude is offered and had no detection tooling at the time.
• EU AI content rules force publishers to label or risk 3% of turnover - The finalised Code of Practice, its 200-token text threshold and the penalty structure.
• Google signs EU AI code as advertisers face 3% turnover fines August 2 - Google's July 2026 signature and the free-detection and zero-retention obligations in Section 1.
• Meta faces 3% turnover fines in 5 days as it signs EU AI content code - Meta's reversal of its earlier position and the two-layer marking requirement.
• EU publishes free AI labelling icons ahead of August 2026 deadline - The deployer-side icons, the dropped provenance-certificate option and the December 2026 and February 2027 deadlines.
• Brussels sets AI labeling rules as data errors quietly drain ad budgets - How Article 50 splits duties between providers and deployers.
• Agencies, not clients, usually carry AI label duty, IAB Austria guide says - Why an invisible watermark does not replace a visible label.
• Google's Lyria 3.5 puts full-length AI songs in Gemini and the API - The provider-side marking versus deployer-side labelling distinction for generated audio.
• EU AI Office opens three complaint routes covering Google and Meta systems - Why ad tech firms building on third-party models count as downstream providers.

Summary

Who: OpenAI, which makes ChatGPT and Codex; API customers worldwide; approved researchers and expert organizations; users on all ChatGPT and Codex plans in the EU; EU regulators enforcing the AI Act.

What: An invisible statistical watermark, called textGrain, for text from ChatGPT and Codex in the EU, opt-in watermarking for select API models worldwide, and a detector restricted to vetted researchers and expert organizations. OpenAI's tests put detection at about 80% for 200-token psychology passages and 17% for 400-token passages after a quarter of the words were swapped for synonyms.

When: Today (October 5, 2026) for API opt-in and detector applications; over the coming weeks for the EU rollout and cloud partners. The EU's conformity deadlines fall on December 2, 2026 and February 2, 2027.

Where: European Union for ChatGPT and Codex; worldwide for API customers who opt in.

Why: OpenAI cited EU regulatory requirements under the AI Act, which expects providers to make generated text machine-readable as AI-generated, while stating that text watermarking has significant technical limits.