badge icon

This article was automatically translated from the original Turkish version.

Blog
Blog
Avatar
AuthorYusuf Erdem ErdemOctober 26, 2026 at 4:00 AM

LLM Watermarking (1/2): Watermarking, Working Principles, and Costs

T3 AI+2 More
Quote
T3 AI Developer
T3 AI Developer

This article is the first of a two-part series. The second article examines what happens when such a watermark is embedded into a model’s training data: “Tracking Traces, Radioactive Models, and Data Contamination.” Both articles can be read independently.


"Every text leaves a trace. The real question is who can read it."


Who wrote this sentence and paragraph? A human or a language model? The sentences flow smoothly, word choices are appropriate, and the thought remains coherent. Today, it is often impossible for most readers to distinguish between text written by a high-quality language model and text written by a careful human. By the end of the first section of this article, we will be able to make a definitive claim about this paragraph—but not through intuition, through statistics.


The same issue arises frequently in everyday life. A teacher wants to know whether an assignment was written by the student themselves. An editor asks whether an explanation provided by a source truly originated from a human. A model developer wonders how much of the data collected from the internet consists of outputs from other models. Current tools, however, remain inadequate: “AI detectors” that analyze writing style produce unreliable results; file metadata is lost upon copying; and invisible characters hidden within text vanish with a single copy-paste operation.


Watermarking approaches this problem from a different angle. It does not add anything to the text after generation; instead, it intervenes in how the text is produced. The goal is to leave a trace that readers cannot see but that someone with a secret key can detect statistically. This method is already in use today: the EU Artificial Intelligence Act’s transparency requirements came into effect in August 2026.【1】 Providers such as Anthropic and Google watermark the outputs of their models,【2】 and open-source inference engines like vLLM also offer watermark support.【3】


In this article, we first explain at which stage of text generation watermarking is applied and how it is detected. We then examine what can be altered in a model’s output and in which contexts it is used. Finally, we test these methods on Turkish data: what is the cost of marking an existing text, and how strong is the trace in text marked during generation? There is also an unexpected but effective application that was not originally envisioned for watermarking: identifying the training data used by a model. This topic is addressed in the second article of the series; at the end of this article, we will point to the door this opens.

Principles of Watermarking

Bir dil modelinin sonraki token seçimi

Next Token Selection by a Language Model (Generated by Artificial Intelligence.)

How Does a Model Choose the Next Word?

At each step, a language model generates a logit (raw score) for every token (the smallest unit of text) in its vocabulary. Softmax converts these logits into probabilities, and then a sampler selects a token based on these probabilities. Suppose the model’s probability distribution after the phrase “Hava bugün soğuk ve” is as follows:


Table 1: Hypothetical Probability Distribution of the Model After the Phrase “Hava bugün soğuk ve”

Watermarking intervenes in this selection from the distribution.

When a selection is made according to probabilities (sampling), the word “kapalı” is chosen with a 35% probability and “gri” with a 30% probability. The core idea of watermarking lies precisely in this final step: intervening in how the selection is made.

Two Approaches: Modifying the Logits or Changing the Selection Step

Most common watermarking schemes use the same method: a secret key and the last k tokens (the watermark window) are fed into a pseudo-random function (PRF). The PRF generates a value for each token in the vocabulary. To someone unaware of the key, these values are indistinguishable from random; to someone who knows the key, they produce the same result every time. Schemes are divided into two categories based on how they use these values.

  • First approach: Modify the logits. The scheme known as KGW (named after the initials of its authors Kirchenbauer, Geiping, and Wen) divides the vocabulary at each step into a green list and a red list using the PRF, then adds a fixed δ to the logits of the green tokens.【4】 In our example, suppose “gri” and “rutubetli” fall into the green list and δ = 2:

Table 2: Effect of KGW

With δ = 2, the probabilities of the green tokens “gri” and “rutubetli” increase while those of the others decrease; the distribution is altered.

The probability of the token “gri” rises from 0.30 to 0.63. The distribution has genuinely changed, so this scheme is called distortionary. As δ increases, the watermark becomes stronger but the text becomes more distorted.

  • Second approach: Modify the selection step. The Gumbel-max scheme【5】 does not alter the logits at all. Instead of using true randomness, it uses the R values generated by the PRF and selects the token with the highest value of R^(1/p):

Table 3: Selection with Gumbel-max

Watermark'ın iki müdahale yolu.

Two Ways Watermark Intervenes (Generated with Artificial Intelligence.)

The token with the highest R^(1/p) value calculated from the PRF’s R values (“gri”) is selected; the original probabilities remain unchanged.


This time, “gray” won. The difference is this: when averaging over the keys, each token is selected exactly according to its probability. Over the long term, the model still says “gray” 30 percent of the time. Therefore, Gumbel-max is a distortion-free scheme. By analogy, this situation is like playing Monopoly not by rolling dice but by using the consecutive digits of pi. The moves appear random in practice, but someone who knows pi could examine the game record and determine that it was played using pi.【6】 SynthID-Text, used by Claude and Gemini, belongs to this family.【7】 Gumbel-max will also be used in the experiments in the “Experiments” section.

How Is the Watermark Read?

 Kanıtın metin uzunluğuyla birikmesi.

Accumulation of Evidence with Text Length (Generated by Artificial Intelligence.)

No external model is needed for detection; only the key and the tokenizer (the component that splits text into tokens) are required. The detector recalculates the PRF value for each token and converts it into a score. In a watermark-free text, the average score for each token is 1. In a watermark-laden text, the selected tokens tend to have higher PRF values and therefore higher scores. Continuing our example, suppose the text is: “The weather today is cold and gray.”


Table 4: Detector Score Calculation

The PRF values of selected tokens are converted into scores; the total score (6.26) exceeds the expected value (4) for four tokens.


The expected total score for four tokens is 4; in this example, it is 6.26. The probability that this result occurred by chance—that is, the p-value—is approximately 0.13: about one in eight watermark-free texts would achieve such a high score by random chance. A single sentence proves nothing. But if the same trend persists across hundreds of tokens in simulation, the p-value drops below one in a million, and if it continues over four hundred tokens, it falls below 10⁻²⁰. The signal accumulates with length.

Entropinin izi belirlemesi.

Trace Determination of Entropy (Generated by Artificial Intelligence.)


How quickly the signal accumulates is determined by entropy. Entropy measures the uncertainty in the model’s probability distribution over the next token: it is low when probability is concentrated on a few tokens and high when it is spread across many. When the model is uncertain, there is room for the watermark to influence the selection. When the model is confident—for example, after typing “2 + 2 =”—only one correct answer remains, and that token carries no watermark. Therefore, the difficulty of detection varies by text type: creative text can be detected in relatively short passages, while code or mathematics requires much longer texts.

During Generation or After?

The method described so far works during generation: a watermark is embedded as the model text is written. What if an existing text, such as a book or a dataset, needs to be marked? In that case, the text is rewritten using a watermarked model. This is called post-hoc watermarking.


The cost of this approach is meaning: the more freedom the rewriting has, the stronger the watermark becomes, but the farther the text drifts from the original. Previous studies have established some findings about this balance: the Gumbel-max family achieves a good compromise between meaning and watermark strength; for a strong watermark, smaller models may be more suitable than larger ones, because larger models produce outputs with lower entropy.【8】Achieving this balance is more difficult in languages other than English and in code. The first question in Turkish experiments arises from this point.【9】

Girişteki iki metnin token skorları.

Token Scores of the Two Texts in the Introduction (Generated by Artificial Intelligence.)

Answer to the Paragraph in the Introduction

The two texts in the introduction, the opening quote and the first paragraph, were rewritten using Qwen3.5-9B with watermarking (temperature, T = 1.0). The opening quote alone consists of 30 tokens: p = 0.02, which is not evidence. The first paragraph has 134 tokens: p ≈ 1×10⁻¹⁰; this is a sufficiently strong value to constitute conclusive evidence. Below, the tokens of both texts are colored according to their scores, as in the first section. The message remains unchanged: a short text is not considered evidence; the watermark accumulates with length.


The main schemes currently in use are variations of the two approaches discussed in this section:


Table 5: Comparison of Major Watermarking Schemes: Step of Intervention, Whether They Are Disruptive, Detection Test, and Application Area


Key insight: A watermark signals the randomness from which words are selected. A detectable signal only accumulates in long texts with sufficient uncertainty.

What Does the Watermark Model Output Change?

The impact of the watermark on quality depends on the scheme used. This section explains at the mechanism level where potential trade-offs arise.


If the distribution changes, quality is at risk. In KGW, the δ added to the green list shifts the model’s probabilities. In most steps this is harmless, as one of several closely ranked good options is selected. But sometimes all appropriate ways of expressing a concept are simultaneously moved to the red list, forcing the model toward a worse token. There is also a subtler risk: alignment training carefully calibrates the model’s probabilities for safe versus unsafe responses, and a watermark that touches the logits can disrupt this calibration. In non-disruptive schemes like Gumbel-max, where the distribution mean is preserved, these risks theoretically vanish.


If the distribution is preserved, diversity decreases. In Gumbel-max, randomness comes from the key, so the same prompt and key always produce the same response. A user seeing only one response will not notice this. But for workflows that generate multiple responses to the same question and select the best one or engage in brainstorming, this is a real loss. An extreme consequence of this determinism is cycles that emerge in repeated patterns:

1 + 1 + → 1 window: (1, +, 1, +)

1 + 1 + 1 → + window: (+, 1, +, 1)

1 + 1 + 1 + → 1 window: (1, +, 1, +) ← repeat


When the window repeats, the PRF produces the same value, the same token is reselected, and generation can become locked in a loop. Two common solutions exist. In the first, when a window encounters the same response a second time, the watermark is skipped for that step and normal sampling resumes. In the second, the system randomly switches between two keys: diversity is restored, but the watermark signal is diluted since it is split across two keys.【15】


Benchmark scores change negligibly, but measurement requires care. In non-disruptive schemes, the average benchmark scores of watermark and non-watermark models are practically identical.【16】 Observed small deviations are often due not to reasoning errors but to minor formatting issues. A known example is code generation: a watermark model may write the correct solution but omit a required `import` line, causing the test to fail.【17】 Since sampling itself affects scores, comparisons based on a single run can be misleading; measuring the watermark’s impact requires repeated evaluations with multiple seeds.【18】 In human evaluations, no significant preference difference has been reported between watermark and non-watermark responses.【19】


The speed penalty is negligible. The watermark activates only after logits are generated; it does not alter the model’s weights or memory and adds no extra tokens. The additional computation required is merely calculating PRF values, which introduces only an imperceptible delay to generation.【20】


Table 6: Comparison of KGW with Gumbel-max/SynthID

KGW alters the distribution and thus carries risks to quality and alignment; Gumbel-max preserves the distribution, with its main cost being reduced diversity.


Table 7: Two Responses to the Same Question

It is not possible to distinguish which one is watermarked by reading; the answer is provided in the paragraph below.

Answer: Answer A was watermarked: N = 162 tokens (N is the number of tokens in the text), −log₁₀(p) = 8.4 (the base-10 logarithm of the p-value; higher values indicate stronger watermarking); the same text generated with incorrect keys yielded values of 0.01 and 0.23. Answer B was unwatermarked: N = 162, −log₁₀(p) = 0.13. This test was repeated across three questions; the −log₁₀(p) values for watermarked answers were 8.4, 12.0, and 8.9, while their unwatermarked counterparts yielded 0.13, 0.20, and 0.69. The two answers were similar in fluency and content; the difference was visible only through the key.


Detail: Costs of disruptive schemes. The WatME study demonstrated that the green/red distinction could collapse all appropriate ways of expressing a concept into red, and proposed a rule that retains at least one synonymous token in green.【21】 Another study by Verma and colleagues found that watermarking methods that intervene in logits also shift the probabilities calibrated through alignment training: models either became less secure but more helpful, or began rejecting requests unnecessarily. This effect persisted even after accounting for general quality loss. Since the study used KGW-type schemes and high temperature settings, the results should be interpreted as indicating that the impact of logit-modifying watermarks on alignment requires separate measurement.【22】


Core inference: Logit-modifying watermarks compromise quality; the primary cost of watermarking that alters the selection step is reduced diversity.

What Is Watermarking For?

While the mechanism and its cost are clear, the central question remains: what claims can this watermark reliably support? The mechanism remains consistent across uses, but its evidentiary value varies significantly depending on context.

AI Text Detection

This is the most well-known application. The actual question watermarking answers is narrow: was this text generated by a model holding this specific key? The absence of a watermark does not prove human authorship; the text could have been produced by a model that does not use watermarking or one that uses a different key. In short texts or factual sentences with a single correct continuation, such as “Newton’s most famous work…,” the signal is weak. In a text where only minor corrections were applied by the model, nearly all words are human-authored, and the watermark clings only to those few edits. Similarly, watermarking cannot distinguish whether a model rewrote a text from scratch or heavily revised it.


Another scenario is paraphrasing. Minor edits do not erase the watermark, but a full paraphrase—where every sentence is rewritten from scratch—does. Between these extremes lie various attack types: paraphrasing the text using another model, translating it into another language and back, and, specifically for KGW, predicting the green/red distinction and altering only a few tokens.【23】


Watermark'lı paragrafın uzun belgede aranması.

Searching for a Watermarked Paragraph in a Long Document (Generated by Artificial Intelligence.)

Therefore, watermarking is meaningful in scalable and collaborative contexts, such as a platform scanning its own content or an editor performing compliance checks. In individual and adversarial contexts, such as accusing a student of plagiarism, it cannot serve as sufficient evidence on its own; the method was not designed for this use.


Another common practical scenario involves an AI-generated paragraph embedded within a longer human-written document. If the entire document is scored at once, the thousands of unwatermarked tokens dilute the signal from the hundreds of watermarked ones. In such cases, improving detection quality may involve analyzing the document in segments—searching within windows of varying lengths to identify the most significant region.

Origin Labeling

The most common contemporary justification for watermarks is legal: the EU Artificial Intelligence Act requires providers to mark machine-generated content in a machine-readable format.【24】 Anthropic explicitly applies its watermark for this purpose and states that the key carries no information about the user, institution, or conversation.【25】 The claim here is whether content has passed through a model; the watermark does not encode information about who produced the content.

Provider Tracking of Its Own Output

A less discussed use case is a provider tracking where its own generated text ends up. Avoiding retraining on its own output and detecting whether a competitor has improperly transferred its outputs to its own model are part of this use case. The latter point is the subject of the second article in this series. So far, the question has been about the origin of the text. But text does not always remain at its final destination; sometimes it enters the training data of another model. We return to this question in the second article; here, we first produce the raw material—the Turkish watermark-labeled data—for that question.


Key Inference: The question a watermark answers is the likelihood that a text originated from a model possessing a specific key; this likelihood strengthens as the text grows longer.

Experiments

Dört deneyden oluşan zincir

Chain Consisting of Four Experiments (Generated with Artificial Intelligence.)

The processes described in the first three sections are tested here on Turkish data. The experiments are designed as a four-stage chain spanning two articles. The first two experiments in this article generate watermark-labeled Turkish data: by retroactively marking existing text (Experiment 1) and by marking during response generation (Experiment 2). The final two experiments in the second article use this data to train models and test for the presence of the watermark. All four experiments use the same question set, enabling comparisons on identical content.

Common Setup

Models: All models are selected from the Qwen3.5 family, which shares the same tokenizer; as shown in the second article, tokenizer incompatibility reduces the power of tests conducted on a model. The watermarking and teacher model is Qwen3.5-9B; for post-hoc rewriting, four sizes from 0.8B to 9B are scanned (Experiment 1). The student models are trained from the Qwen3.5-2B base model, as detailed in the second article. No model exceeds 10 billion parameters.


Data: Five thousand question-answer pairs are selected from the open-source Turkish instruction dataset InstrucTurca;【26】 additionally, 500 answers not used in training are reserved for loss measurement, and 300 answers are set aside for Experiment 1 scanning. This dataset is synthetic, created by machine translating English instruction sets; what matters here is that the answers are existing texts never marked with the key. The answers average 282 tokens in length; the total dataset is approximately 1.4 million tokens. The watermark is always applied to the answers, since loss is computed only over the answers during training.


Watermark: Gumbel-max with window size k = 2. As shown in the second article, the effect of k varies with data size; this dataset is larger than a small evaluation test but smaller than large instruction sets, making k = 2 a balanced choice for this intermediate range.【27】 To prevent the loop issue described in Section 2 and preserve the scheme’s non-disruptive property, repeated windows within the same answer are skipped for watermarking.


Key: Twenty candidate keys are tested on watermark-free Turkish text, and the key whose p-value distribution most closely matches a uniform distribution is selected (Kolmogorov-Smirnov test). This step is especially important for Turkish because its agglutinative structure causes certain token patterns to be repeated excessively.


Detection: All measurements in this article are performed on the text: each token’s PRF value is recomputed using the key and tokenizer, and the total score is converted to a p-value (Section 1). Each window within the same answer is scored once (deduplication). Detection directly on the model itself (reading mode) and model-specific additional deduplication are the subjects of the second article.


Table 8: Experiment Steps and Estimated Durations (1 H100)

The total time is approximately 1.5 hours.

Experiment 1: Watermarking Existing Text

Question: Can a Turkish text be watermarked after generation without altering its meaning? What is the cost in terms of meaning loss?


Setup: The source texts are original responses from InstrucTurca. First, a scan is performed on a subset of 300 responses. Variables include the size of the rewriting model (Qwen3.5 versions: 0.8B, 2B, 4B, and 9B) and temperature (0.7, 1.0, 1.2). For each rewrite, three metrics are measured: watermark strength (−log₁₀(p)), semantic similarity (using the multilingual-e5-large embedding model)【28】 and length ratio. Outputs falling outside 75% to 125% of the original length are discarded. Long responses are split into segments and rewritten sequentially, with each segment given the previous segment as context. After analyzing the scan results, the configuration that best balances meaning preservation and watermark strength is selected, and all data is rewritten using this setting (post-hoc data). A single watermark-free rewrite is also performed using the same setting (rewrite control); this data will be needed in the second paper.


Expectation: The known trends in English are expected to hold in Turkish as well, but at a higher cost. That is, smaller models will produce stronger watermarks, larger models will preserve meaning better, and the amount of meaning loss required to achieve the same watermark strength will be greater than in English.

Anlam benzerliği ve iz gücü dengesi.

Balance of Semantic Similarity and Traceability (Generated by Artificial Intelligence.)

How to Interpret the Results?

  • If smaller models produce stronger watermarks: Watermarks only leave traces where the model genuinely makes choices. Larger models write more confidently: they preserve meaning well but leave little room for the watermark to operate; smaller models explore more options, strengthening the watermark at the cost of meaning. Thus, the largest model is not necessary for watermarking.
  • If no configuration yields both strong watermark and high semantic similarity: This implies that post-hoc watermarking in Turkish is more costly than in English. A likely reason is the tokenizer: agglutinative structure splits words into multiple tokens, and suffixes at word ends have low entropy, leaving little room for the watermark to act. The watermark can only be introduced via temperature, at the cost of text distortion.
  • If semantic similarity is high but the text is distorted: Embedding similarity does not measure fluency or grammatical integrity; in Turkish, morphological distortions may escape this metric. Before concluding that meaning is preserved in watermarked text at high temperatures, the text must be read manually.


Watermark'lı yeniden yazım örnekleri.

Watermarked Rewriting Examples (Generated by Artificial Intelligence)

Findings: Temperature was the primary determinant of watermark strength. At T = 0.7, the median −log₁₀(p) was below 3 for all four models (no watermark); at T = 1.0, it ranged from 8 to 25; at T = 1.2, it ranged from 66 to 251. At the same temperature, smaller models produced stronger watermarks (at T = 1.2: 0.8B: 251, 2B: 178, 4B: 142, 9B: 106); semantic similarity increased in the opposite direction, from 0.874 to 0.937. The first half of the expectation was confirmed.


The second half of the expectation was also confirmed, even more strongly than predicted. The configuration yielding the highest proportion of rewrites that were simultaneously similar (≥ 0.85), within length bounds, and detectable (−log₁₀(p) ≥ 6) was selected: Qwen3.5-9B, T = 1.2 (proportion: 53%; median −log₁₀(p) = 106, similarity: 0.937, length transition: 47%). This represents a deviation from the planned criterion and is explicitly documented.


Pre- and post-example pairs reveal aspects of this cost not captured by numbers. At high temperatures, rewrites preserve the original phrasing in most sentences; however, in some sentences, fluency is lost (suffixes and word boundaries become distorted). Embedding similarity is insensitive to these subtle distortions: in two examples, similarity remains high (0.96 and 0.94), yet a human reader immediately detects the difference. Therefore, meaning preservation must be evaluated not only numerically but also by reading. These two examples are not typical but optimistic selections: they were chosen from candidates with length ratios between 0.90 and 1.10, N ≥ 60, and −log₁₀(p) ≥ 6, specifically those closest to the original at the word level (for the first example, similarity ≥ 0.95; for the second, 0.85–0.95).


Table 9: Experiment 1 Scan Results (by Model and Temperature)

As temperature increases, watermark strength increases but semantic similarity decreases; at the same temperature, smaller models produce stronger watermarks.


Length filter and training: When all 5,000 responses were rewritten using the selected settings, 2,141 responses (42.8%) fell outside the length band. The rate was similar across categories: creative 42%, factual 42%, math/code 47%. The deviation was one-directional: rewritten responses were often longer than the originals (median length ratio 1.13, 90th percentile 1.78). The filtered responses were not removed from the training data in the second writing experiment; post-hoc conditions were trained on all rewrites prior to filtering. This choice affects both the trace and quality of the post-hoc data and must be kept in mind during interpretation.

Experiment 2: Watermark During Generation

Question: When the teacher model generates watermarked responses to the same questions, how strong is the watermark, and how does it differ from the post-hoc method?


Setup: Qwen3.5-9B generates one watermarked response (generation data) and one unwatermarked response (clean data) for each of the 5,000 questions. Questions are grouped by category: open-ended and creative, factual, and math or code. Measurements include: detection power by group and token count; diversity (measured by the rate of identical responses to the same 200 questions and by the proportion of unique two-word sequences across all responses, distinct-2); response length; and a short quality control test of 100 questions from the 【29】 Turkish middle school mathematics problems of the TÜBİTAK Science Olympiad: the model solves each question and provides the answer enclosed in \boxed{}, with accuracy compared between watermarked (four different keys) and unwatermarked (four different seeds) generations.


Expectation: The “length × entropy” relationship observed in Section 1 is expected to hold in this data: creative responses will be reliably detected within a few hundred tokens, while math and code responses will lag significantly. More interesting is the comparison with post-hoc data. In post-hoc rewriting, the model is constrained to remain faithful to the original, leaving less freedom of choice. Therefore, for the same content, the post-hoc data is expected to yield a weaker watermark per token.

Üretim sırasında iz gücünün token sayısıyla birikimi.

Accumulation of Production Power Tokens During Manufacturing (Generated by Artificial Intelligence.)

How Are the Results Interpreted?

  • When there is a large difference between topics: The watermark depends on how free-form the response is. In numerical and code content with a single correct answer, options are limited, requiring longer text for equivalent strength. When the difference is small, the watermark depends more on length than topic; sufficient length allows all content to be detected.
  • If post-hoc watermarking is weak: This is expected. Since the rewriting model must remain faithful to the original, its freedom of choice is restricted. If the post-hoc watermark is strong, it is typically because a higher temperature was used; the watermark strength may have been achieved at the cost of meaning and fluency.
  • If the same answer always appears for the same question: Gumbel-max uses the same random number in the same context, producing identical responses to identical prompts. Users who see only one response do not notice this; workflows generating multiple responses to the same prompt are affected. If diversity is preserved, the cost is limited only to repeated prompts.


Token başına skor dağılımı.

Score Distribution per Token (Generated by Artificial Intelligence.)

The difference between generation and post-hoc is substantial. The average score per token is 3.45 in post-hoc data, 1.35 in generation data, and 1.01 in clean data (chance level is 1.0). The deviation from chance is about seven times larger in post-hoc data (2.45 vs 0.35); the median −log₁₀(p) reflects this similarly: 106 vs 7.6. Both generation and post-hoc data were produced using the same model (Qwen3.5-9B), differing only in temperature (T = 0.8 and T = 1.2). Whether this difference transfers to the model itself is the central question of the second experiment.


Findings: The median watermark in generated responses is 7.6 (−log₁₀(p); interquartile range 3.9–11.8), compared to 0.33 in clean data. The accumulation is consistent: at 25 tokens the median is 1.3, at 100 tokens it is 3.4, and at 400 tokens it is 11.2. At 400 tokens, 91% of creative responses, 96% of factual responses, and 83% of math/code responses exceed the p < 10⁻⁶ threshold; none in clean data. The difference between topics was smaller than expected: at 100 tokens, creative responses scored 4.0, factual 3.3, and math/code 2.8; at 400 tokens, factual and creative responses are nearly equal (12.0 and 11.9), with math/code lagging behind (9.1). It was anticipated that math/code would fall significantly behind; the difference remained moderate.


Table 10: Comparison of Generated, Post-hoc, and Watermark-free (Clean) Data

Post-hoc watermarking is stronger; generation data preserves quality and corpus diversity, but identical prompts always produce identical outputs.


Diversity: When two responses were generated for each of 200 prompts, 100% of watermark-embedded generations (same key) produced identical outputs; in watermark-free generations (different seeds), this rate was 1%. Gumbel-max uses the same random number in the same context, resulting in identical outputs for identical prompts. However, corpus-level diversity is preserved: distinct-2 is 0.649 in generation data and 0.663 in clean data. Thus, the cost is visible only in workflows that repeat the same prompt.


Quality control: For TÜBİTAK middle school math questions, accuracy was 100% in both watermark and non-watermark generations; no difference. Approximately 11% of answers (10% in non-watermark) were cut off at the generation length limit; since this rate was identical under both conditions, the comparison is valid, but absolute accuracy is therefore low. The measurement is limited to 100 questions and cannot detect small differences; instead of saying “quality did not degrade,” it is more accurate to say “no degradation was observed at this resolution.” The question set consists of the middle school mathematics section of TÜBİTAK Olympiad problems.


Summary of the experiments at a glance:

Table 11: Objective and One-Sentence Result of Experiment 1 and Experiment 2

Frequently Asked Questions

  • Does the watermark slow down or increase the cost of the model?

No. The watermark only intervenes in the step of selecting the next token; it does not modify the model itself or generate additional tokens. Therefore, its impact on speed is negligible and does not change the cost of running the model.【30】

  • How does watermark detection differ from commercial AI detection tools (e.g. Pangram)?

Companies offering such tools do not have access to the model provider’s secret key. Therefore, they analyze the text itself, looking for stylistic clues such as patterns common in AI-generated text or overuse of certain words. Watermark detection, by contrast, does not rely on style; it statistically tests whether token selections are consistent with a hidden key. The two methods answer fundamentally different questions.

  • Does watermark detection provide certainty?

No, results are always probabilistic. The watermark can only indicate that a particular model likely played a major role in generating the text; it cannot distinguish whether the model wrote the text outright or was heavily edited by a human.【31】 It may miss watermarked text in short passages, low-entropy factual sentences, or texts rewritten from scratch. Conversely, there is also a false positive risk: even with a very small p-value, there is always a non-zero probability that a non-watermarked text appears meaningful by chance. Therefore, watermark results should be interpreted not as definitive judgments on their own, but as indicators to be evaluated alongside other evidence.

Conclusion

What watermarking does appears modest at first glance: it replaces the model’s own random selections with values derived from a secret key. In modern architectures, the cost to quality is negligible, the cost to speed is comparable to noise, and the primary cost is to diversity.


The payoff for this modest intervention is equally modest. It does not provide a definitive answer to the question “Was this text written by AI?”; it fails in short texts, low-entropy texts, or texts written from scratch. The paragraph above was a small example of this: a single word proved nothing, but the entire paragraph did.


A paragraph from the experiments: The findings of Section 4 concretized the above framework for Turkish. Post-hoc watermarking left a strong signal (median −log₁₀(p) = 106), but this was achieved at the cost of high temperature—that is, some loss of fluency. Watermarking during generation was cleaner but required approximately 400 tokens (median −log₁₀(p) = 7.6). Smaller models left stronger signals but preserved meaning less well. These results are specific to a single setup, a single language, and a single model family; replication in other languages and schemes is necessary to generalize. Knowing under which conditions a method works is more useful than merely knowing that it works.


The signal resides in the text. This raises the question: when this text enters a model’s training data, does the model carry forward the watermark, and if so, can this be proven without access to the original training data? The two datasets generated in this paper—Turkish data watermarked post-hoc and during generation—serve as the input for the second article in this series: “LLM Watermarking (2/2): Tracing Signals, Radioactive Models, and Data Contamination”.

Bibliographies

Altinok, Duygu. “InstrucTurca: Diverse Instructional Content Dataset for Turkish.” Hugging Face. 2024. Accessed October 15, 2026.https://huggingface.co/datasets/turkish-nlp-suite/InstrucTurca

Anthropic. “How Claude's text watermark works.” August 14, 2026. Accessed October 5, 2026.https://anthropic.com/news/claude-text-watermark

Chen, L., Y. Bian, Y. Deng, et al. “WatME: Towards Lossless Watermarking Through Lexical Redundancy.” *arXiv* 2023. Accessed October 4, 2026.https://arxiv.org/abs/2311.09832

Dathathri, S., A. See, S. Ghaisas et al. “Scalable Watermarking for Identifying Large Language Model Outputs.” *Nature* 634 (2024): 818–823. Accessed September 30, 2026.https://www.nature.com/articles/s41586-024-08025-4

EU Artificial Intelligence Act. “Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems.” Transparency Obligations for Providers and Deployers of Certain AI Systems. Accessed October 2, 2026.https://artificialintelligenceact.eu/article/50/

Fernandez, P., T. Sander, H. Elsahar et al. “How Good is Post-Hoc Watermarking With Language Model Rephrasing?” *arXiv* 2025. Accessed October 12, 2026.https://arxiv.org/abs/2512.16904

Fu, J., X. Zhao, R. Yang. “GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick.” *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics* 1. (2024): 5791–5808. Accessed October 7, 2026.https://aclanthology.org/2024.acl-long.315/

Kirchenbauer, J., J. Geiping, Y. Wen, et al. “A Watermark for Large Language Models.” *arXiv* 2023. Accessed October 1, 2026.https://arxiv.org/abs/2301.10226

Krishna K., Y. Song, M. Karpinska et al. “Paraphrasing Evades Detectors of AI-Generated Text, but Retrieval is an Effective Defense.” *Advances in Neural Information Processing Systems* 36 (NeurIPS 2023): 1–32. Accessed October 6, 2026.https://neurips.cc/virtual/2023/poster/71402

Kuditipudi, R., J. Thickstun, T. Hashimoto, and P. Liang. “Robust Distortion-Free Watermarks for Language Models.” *arXiv* 2023. Accessed October 9, 2026.https://arxiv.org/abs/2307.15593

Qwen. "Qwen3.5: Towards Native Multimodal Agents." Qwen. 2026. Accessed October 14, 2026.https://qwen.ai/blog?id=qwen3.5

Rialland, R., S. Veitner, and T. Ruiz. “Watermarking in vLLM.” *vLLM Blog*. 2026. Accessed October 8, 2026.https://vllm.ai/blog/2026-09-24-watermarking-in-vllm

Sander, T., H. Chang, T. Soucek, et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection.” *arXiv* 2026. Accessed October 3, 2026.https://arxiv.org/abs/2605.12456

Sander, T., P. Fernandez, S. Mahloujifar et al. “Detecting Benchmark Contamination Through Watermarking.” *arXiv* 2025. Accessed October 10, 2026.https://arxiv.org/abs/2502.17259

Verma, A., N. Phan, and S. Trivedi. “Watermarking Degrades Alignment in Language Models: Analysis and Mitigation.” *arXiv* 2025. Accessed October 11, 2026.https://arxiv.org/abs/2506.04462

Wang, L., N. Yang, X. Huang, et al. “Multilingual E5 Text Embeddings: A Technical Report.” *arXiv* 2024. Accessed October 2, 2026.https://arxiv.org/abs/2402.05672

Wu, Qilong, and Varun Chandrasekaran. “Bypassing LLM Watermarks with Color-Aware Substitutions.” *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics* 1. (2024): 8549–8581. Accessed October 13, 2026.https://aclanthology.org/2024.acl-long.464/

Yıldız Teknik Üniversitesi COSMOS Araştırma Grubu. “TUBITAK Science Olympiad Dataset.” Hugging Face. Accessed October 8, 2026.https://huggingface.co/datasets/ytu-ce-cosmos/tubitak-science-olympiad-tr

Citations

  • [1]

    EU Artificial Intelligence Act, “Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems,” Transparency Obligations for Providers and Deployers of Certain AI Systems, Accessed October 2, 2026.

  • [2]

    Anthropic, “How Claude’s Text Watermark Works”, Accessed October 5, 2026. ; S. Dathathri, A. See, S. Ghaisas et al., “Scalable Watermarking for Identifying Large Language Model Outputs,” *Nature* 634 (2024), 819, Accessed September 30, 2026.

  • [3]

    R. Rialland, S. Veitner, and T. Ruiz, “Watermarking in vLLM.” *vLLM Blog*, 2026. Accessed October 8, 2026.

  • [4]

    J. Kirchenbauer, J. Geiping, Y. Wen, et al. “A Watermark for Large Language Models.” *arXiv* 2023, 3-4. Accessed October 1, 2026.

  • [5]

    J. Fu, X. Zhao, R. Yang, “GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick,” *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics* 1 (2024), 5791–5808. Accessed October 7, 2026.

  • [6]

    Anthropic, “How Claude’s Text Watermark Works.”

  • [7]

    S. Dathathri et al. “Scalable Watermarking for Identifying Large Language Model Outputs,” 819; Anthropic, “How Claude's text watermark works.”

  • [8]

    T. Sander, P. Fernandez, S. Mahloujifar et al. “Detecting Benchmark Contamination Through Watermarking,” *arXiv* 2025, 7-8. Accessed October 10, 2026.

  • [9]

    T. Sander, P. Fernandez, S. Mahloujifar et al., 10–12.

  • [10]

    J. Kirchenbauer, J. Geiping, Y. Wen, et al. “A Watermark for Large Language Models.”

  • [11]

    J. Fu, X. Zhao, R. Yang, “GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick”

  • [12]

    S. Dathathri et al. "Scalable Watermarking for Identifying Large Language Model Outputs."

  • [13]

    T. Sander, H. Chang, T. Soucek, et al. "TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection." *arXiv* 2026. Accessed October 3, 2026.

  • [14]

    R. Kuditipudi, J. Thickstun, T. Hashimoto, and P. Liang, “Robust Distortion-Free Watermarks for Language Models,” *arXiv* 2023, Accessed October 9, 2026.

  • [15]

    T. Sander, H. Chang, T. Soucek et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection,” 4-6; R. Rialland, S. Veitner and T. Ruiz, “Watermarking in vLLM,” *vLLM Blog*.

  • [16]

    Sander, T., Chang, H., Soucek, T., et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection,” 35-36.

  • [17]

    Sander, T., Chang, H., Soucek, T., et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection,” 9.

  • [18]

    Sander, T., Chang, H., Soucek, T., et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection,” 35-36.

  • [19]

    Sander, T., Chang, H., Soucek, T., et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection,” 9-10, 37.

  • [20]

    T. Sander, H. Chang, T. Soucek, et al. “TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection,” 13; S. Dathathri, et al., 821-822.

  • [21]

    L. Chen, Y. Bian, Y. Deng et al. "WatME: Towards Lossless Watermarking Through Lexical Redundancy." *arXiv* 2023, 1, 5. Accessed October 4, 2026.

  • [22]

    A. Verma, N. Phan, and S. Trivedi, “Watermarking Degrades Alignment in Language Models: Analysis and Mitigation,” *arXiv* 2025, 1-9. Accessed October 11, 2026.

  • [23]

    Q. Wu and V. Chandrasekaran, “Bypassing LLM Watermarks with Color-Aware Substitutions,” *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics* 1 (2024): 8549–8554. Accessed October 13, 2026. ; K. Krishna, Y. Song, M. Karpinska et al., “Paraphrasing Evades Detectors of AI-Generated Text, but Retrieval is an Effective Defense,” *Advances in Neural Information Processing Systems* 36 (2023): 2, 6. Accessed October 6, 2026.

  • [24]

    EU Artificial Intelligence Act, “Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems.”

  • [25]

    Anthropic. “How Claude's text watermark works.”

  • [26]

    Duygu Altinok, “InstrucTurca: Diverse Instructional Content Dataset for Turkish,” Hugging Face, 2024. Accessed October 15, 2026.

  • [27]

    T. Sander, P. Fernandez, S. Mahloujifar, et al. “Detecting Benchmark Contamination Through Watermarking,” 8; P. Fernandez, T. Sander, H. Elsahar, et al. “How Good is Post-Hoc Watermarking With Language Model Rephrasing?” *arXiv* 2025, 5. Accessed October 12, 2026.

  • [28]

    L. Wang, N. Yang, X. Huang, et al. "Multilingual E5 Text Embeddings: A Technical Report," *arXiv* 2024, Accessed October 2, 2026.

  • [29]

    Yıldız Teknik Üniversitesi COSMOS Araştırma Grubu. “TUBITAK Science Olympiad Dataset.” Hugging Face, Accessed October 8, 2026.

  • [30]

    R. Rialland, S. Veitner, and T. Ruiz, “Watermarking in vLLM.” *vLLM Blog*.

  • [31]

    Anthropic, “How Claude's text watermark works.”

T3 AI
Türkiye's Large Language Model,is growing together with you.

Contents

  • Principles of Watermarking

    • How Does a Model Choose the Next Word?

    • Two Approaches: Modifying the Logits or Changing the Selection Step

    • How Is the Watermark Read?

    • During Generation or After?

    • Answer to the Paragraph in the Introduction

  • What Does the Watermark Model Output Change?

  • What Is Watermarking For?

    • AI Text Detection

    • Origin Labeling

    • Provider Tracking of Its Own Output

  • Experiments

    • Common Setup

    • Experiment 1: Watermarking Existing Text

      • How to Interpret the Results?

    • Experiment 2: Watermark During Generation

      • How Are the Results Interpreted?

  • Frequently Asked Questions

  • Conclusion