Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

“We use two settings: DIPPER-1, with lexical diversity set to 60 and order diversity 0 ; and DIPPER-2, with lexical diversity 60 and order diversity 20 .”

“In contrast, Dipper, the most effective attack in the early experiments, performs poorly on this task, achieving only 9 .”

“After generation, post-hoc methods such as para-phrasing (Krishna et al., 2023), or replacing certain characters or words are very effective as well, yet they can significantly alter the original text (Macko et al., 2024).”

“In contrast, on the web, all tested platforms preserved all 130 characters, although some were returned in defanged form in the page source.”

“…fine-tuning a pre-trained language model on a constructed dataset containing both human-written and LLM-generated content; 2) Zero-shot methods [23, 30, 31, 58], which leverage intrinsic properties of typical LLMs, such as probability curves [31] and N-gram divergence [58], for self-detection;…”

“Lastly, we evaluate the robustness of each method against the decoherence attack (Bao et al., 2024) and the paraphrasing attack (Krishna et al., 2023) (see Appendix B.2 for more details) in Figure 4e and Figure 4f.”

“This aligns with recent trends in robustness evaluations (Hans et al., 2024; Krishna et al., 2023; Soto et al., 2024).”

“Regardless of ChatGPT, Krishna et al. [25] also trains a specialized language model called DIP-PER for text paraphrasing, which can rewrite and reorder text based on specified control conditions.”

“Others opt for transforming entire texts through paraphrasing [17], [18].”

“In addition to the substitution attack, we also evaluate the robustness of our proposed method against paraphrasing attack using DIPPER [Krishna et al., 2023].”

“However, score-based methods often depend on specific generators or semantic domains, and known traces may be easy to remove through common transformations such as resizing or compression for images or paraphrasing for text [Gragnaniello et al., 2021, Krishna et al., 2023].”

“Dipper is a paraphrasing model (Krishna et al., 2023).”

“Additionally, we use DIPPER for Recursive Paraphrase (Sadasivan et al., 2023).”

“Even though these methods have been designed to be more robust against such paraphrase attacks (Kirchenbauer et al., 2023b), these paraphrase attacks often use unwatermarked LLMs, e.g. DIP-PER (Krishna et al., 2023) and GPT-3.”

“…Due to the increasing use of tools like ChatGPT as writing assistants, the field of machine-generated text detection has become of high interest within the research community and is being studied extensively et al., 2023; Krishna et al., 2023; Mitchell et al., 2023; Mireshghallah et al., 2023).”

“Figure 6: The tradeoff between AUROC and pass@1 of detecting real and generated samples of HumanEval and MBPP datasets.”

“However, this technique needs a balance between imperceptibility and detectability, while also addressing potential attacks on the watermarks to ensure the integrity of the detection process [23].”

“Krishna et al. (2023) provide a critical base-line by demonstrating that controlled paraphrasing can significantly undermine the performance of AI-generated text detectors while maintaining semantic integrity.”

“Following standard evaluation protocols, we configure DIPPER with lexical diversity L = 20 and order diversity O = 0 .”

“Krishna et al. [10] demonstrated that paraphrasing attacks can evade several detector families and proposed retrieval-based defenses, directly motivating the need to assess cues under paraphrase and revision conditions rather than only on original generations.”

“Although textual style is a distinguishable feature of AI writing (Krishna et al. 2023), this stylistic dependency is particularly problematic in the peer review context, as it risks both unjustly penalizing legitimate AI assistance and overlooking deceptively generated reviews.”

“Watermark-based detectors jeopardize text quality and are extremely vulnerable to paraphrasing attacks [8, 14].”

“…[26], referred to as WMremoval , which removes WMs by perturbing quality-preserving WM outputs to produce high-quality non-WMed outputs; 2) Dipper [93], which paraphrases paragraphs through content reordering and lexical changes without appreciably modifying the text semantics, and 3)…”

“Further, there has been a large-scale exploration of the susceptibility of watermarks to natural paraphrasing attacks and strong adversarial attacks Krishna et al. (2023); Sadasivan et al. (2023); Zhang et al. (2023).”

“…closely mimics human writing, thereby effectively deceiving state-of-the-art detectors (Wang, Chen, Liu, Chen, Chen, Zhang and Cheng, 2025); DIPPER (Krishna, Song, Karpinska, Wieting and Iyyer, 2023) exposed the vulnerability of current AI-generated text detectors when confronted with paraphrasing…”

“In a broader context, our work is related to measuring paraphrases and entailment, both long-standing topics of interest in the NLP and ML (Bhagat and Hovy, 2013; Bowman et al., 2015; Zhou and Bhat, 2021; Opitz et al., 2023; Gubelmann et al., 2023; Krishna et al., 2023).”

“Figure 7 illustrates the accuracy-fairness tradeoffs achieved by our proposed FairOPT algorithm compared to the Static and ROCFPR baselines.”

“Krishna et al (2024) fine-tune LLMs with paraphrased text data to produce text with a similar paraphrasing style.”

“For the latter category, we implement nine attack methods including contraction, lowercase, expansion, misspelling, typo which are proposed by Liang et al. [38], modify, synonym which are proposed by Piet et al. [57], paraphrase [33] and translation [13].”

“Krishna et al. (2023) introduced DIP-PER, an 11b paraphrase generation model that can evade several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI’s now defunct text classifier (Tian, 2023; Mitchell et al., 2023; Kirchner et al., 2023).”

“Various effective attacks, i.e. paraphrasing the output directly (Krishna et al., 2023; Sadasivan et al., 2023), paraphrasing the input (Ha et al., 2023; Shi et al., 2024), and concatenating deceptive in-context examples to the input (Shi et al., 2024; Lu et al., 2024) could drop the detection…”

“Krishna et al. (2023) follow the recording approach and Kirchenbauer et al. (2023) show possibilities of watermarking during generation.”

“Furthermore, edits are associated with relatively short pieces of “text atoms” like sentences whereas the signal discriminating a GLM from a human in a short text piece may be faint or not exist [ZHR + 19, KSK + 23, JHN23, Els23].”

“Several previous works have studied the effects of automated paraphrasers (Krishna et al., 2023; Russell et al., 2025; Sadasivan et al., 2025; Cheng et al., 2025) and “humanizers” (Masrour et al., 2025) on how they degrade AI-generated text.”

“Paraphrasing Evades Detectors of AI-generated Text (Krishna et al., 2023).”

“To assess robustness, we evaluate against SOTA removal and model stealing attacks: 1) Dipper [47] (sentence-level paraphrasing); 2) Substitution [69] (token-level synonym replacement via WordNet [63]); 3) SIRA [14] (self-information–based paraphrasing); and 4) model stealing [5, 53, 80, 16]…”

“• DIPPER [Krishna et al., 2023]: A supervised paraphrasing method that aims to preserve meaning while varying surface form, using lexical and sentence-level controls to steer diversity.”

“Common threats include evasion , where outputs are modified to remove the watermark [Krishna et al., 2023, Diaa et al., 2024, Pang et al., 2024b], forgery , where non-watermarked content is attributed to a watermarked source [Aremu et al., 2025, Jovanovi´c et al., 2024, Gloaguen et al., 2024,…”

“5 shows that, for all methods, the token distribution entropy decreases monotonically for each layer.”

“Empirical evidence suggests that state-of-the-art detectors often achieve accuracy below 80% and perform unevenly across languages, formats, and adversarial paraphrasing (Weber-Wulff et al. 2023, Sadasivan et al. 2023, Kalpesh et al. 2023).”

“In compliance with the mandatory data availability requirements, the reproduction package, including the source code and the experimental data, is publicly available at: https://github.com/Wangvje/ AURA.”

“In practice, while methods like paraphrasing [Krishna et al., 2023] can often destroy hidden messages, advanced steganographic techniques have emerged which survive such interventions [Mathew et al., 2024].”

“Since AUROC masks performance at low false positive rates (FPR), which are critical in generation detection tasks (Carlini et al. 2022; Krishna et al. 2023; Yang et al. 2024), we also report the true positive rate (TPR) at a fixed FPR of 1% (Roberts et al. 2024).”

“One is retrieval-based schemes [16] that require accessing and storing all the texts generated during the interaction process of LLM.”

“Studies on how to attack these detectors are also being conducted in different views, such as paraphrasing [Sadasivan et al., 2023, Krishna et al., 2023], adversarial prompting [Tufts et al., 2024, Cheng et al., 2025a], and other forms of authorship obfuscation [Macko et al., 2024].”

“After which, we conducted paraphrase attack on the watermarked texts using the Dipper model [15], we followed previous work’s [13] parameter settings, using a lex diversity of 60.”

“Dipper (Krishna et al., 2023) paraphrasing at 20% diversity reduces Predictive-Bidirectional to 72 .”

“We conduct a comprehensive evaluation of MPAC, RSBH, and our two proposed methods, MajorMark and MajorMark + , on the OpenGen (Krishna et al. 2023) and Essays (Schuhmann 2023) datasets.”

“Existing evasion methods typically involve content modification: paraphrasing attacks alter semantic meaning (Krishna et al., 2023), character sub-stitution techniques replace visually similar characters (Creo and Pudasaini, 2025), and adversarial perturbations introduce subtle textual changes…”

“We report four calculated thresholds by classifier: TPR@FPR=5% [6], TPR@FPR=1%[114], recall@precision=95%, and recall@precision=99% in Table 20.”

“For LLMs deployed in security roles (e.g., spam filtering), the session covers evasion attacks using techniques like style-transfer [5] or paraphrasing [3] to bypass classifiers.”

“However, techniques such as paraphrasing and prompt tuning challenge many of the current LLM detectors, suggesting the general intractability of combating adversarial techniques at scale [Krishna et al., 2023, Kumarage et al., 2023, Lu et al., 2024, Sadasivan et al., 2025].”

“We choose 100 samples from Reddit generated by each of Detector-Guided DPO, DIPPER, OUTFOX, and our style-aware-paraphraser at random.”

“Therefore, we test the degree of bias towards non-native speakers in TempTest, following a setup of Tulchinskii et al. (2024), and robustness to paraphrasing attacks using Dipper (Krishna et al., 2024).”

“Yet, these detectors can be easily fooled by simple paraphrasing (Krishna et al. 2024) and are not robust to unseen models and domains (Weber-Wulff et al. 2023).”

“Krishna et al. (2024) includes semantic and adaptive watermarking and recording-based methods to improve robustness.”

“We follow the methodology of [5], [13], [25] by using TPR at 1% FPR as our primary metric.”

“One group of researchers showed that having the AI-generated text detector use semantic retrieval increases the detector ’ s robustness in the face of paraphrasing (Krishna et al., 2024).”

“…et al., 2019; Uchendu et al., 2020), primarily using RoBERTa (Liu, 2019), provide more accurate results than statistical methods (Li et al., 2024; Liu et al., 2023b), but lack robustness (Li et al., 2024; Koike et al., 2024; Krishna et al., 2024; Chakraborty et al., 2023; Tulchinskii et al., 2024).”

“(ii) We simulated paraphrasing by employing DIPPER (Krishna et al., 2023), a fine-tuning model for watermark evasion, to vary lexical (L) and order (O) diversity.”

“To evaluate robustness, we consider two forms of attacks: (1) Substitution , where tokens are randomly replaced with synonyms; and (2) Dipper , using the Dipper para-phraser (Krishna et al., 2023) to rewrite the text.”

“By quantifying distributional discrepancies in linguistic patterns between the input and reference samples (human or machine), these methods significantly enhance generalization in cross-domain scenarios (Krishna et al., 2023; Sadasivan et al., 2025).”

“(Krishna et al., 2024) train a generative model (DIPPER) to evade detection.”

“Also, we use paraphrasing to humanize A.I.-generated text [11].”

“Furthermore, recent studies (Sadasivan et al., 2023) (Krishna et al., 2023) have shown that watermarking schemes can easily be bypassed using another model that can paraphrase the output of an LLM.”

“Most text watermarks operate through token choices, which paraphrasing can remove at a high ¯ q [12,26].”

“Para-phrastic rewriting remains challenging because it breaks the fixed tape readout (Krishna et al., 2023; Hou et al., 2024).”

“Attribution faces paraphrasing with Dipper [11] and round-trip translation with opus-mt [34].”

“A paraphrasing mitigation [15] tests whether simply rewriting messages is enough to break the channel.”

“Prior work shows that detector performance is sensitive to paraphrasing, rewriting, adversarial prompting, and benchmark construction (Krishna et al., 2023; Sadasivan et al., 2025; Shi et al., 2024; Dugan et al., 2024).”

“Consider the shifts common in LLM detection: adversarial humanization, in which users or humanizer services (Grammarly, 2026; Humanize AI, 2026; Krishna et al., 2023) strategically rewrite AI-generated text to evade a published detector; and natural shifts over time, due to new LLM releases and…”

“Notably, AliMark outperforms baselines under most para-phrasing attacks, and the performance gain becomes much more significant under stronger paraphrasers like DIPPER (Krishna et al., 2023) and GPT-3.”

“Robustness is measured as post-attack TPR across seven attacks: DIPPER [Krishna et al., 2023], random walk (T5-XL span masking, 50 steps) [Zhang et al., 2023], context synonym substitution, synonym substitution, word deletion, word substitution, and sentence reordering as implemented in MarkLLM…”

“…(i) substitutions , including synonym replacement, back-translation, and model-based paraphrasing (Morris et al., 2020; Wieting & Gimpel, 2018; Krishna et al., 2023), (ii) insertions/deletions that break token–bit align-2 Low extraction accuracy (49.2% @ 32 bits); no calibrated detection (Qu…”

“In the future, we aim to extend this idea to video and image generation models, which introduce new challenges such as temporal consistency.”

“We construct the training set from C4-RealNewslike subset [25] and the test set from four datasets: Reddit WritingPrompts [26], LFQA [27], and the BookReport and FakeNews subsets of MMW [28], following [11].”

“Paraphrase-based attacks [Krishna et al., 2023], style transfer, and direct human editing have all been demonstrated to substantially reduce detector accuracy.”

“Post-hoc detection approaches [27, 36, 12, 21, 28].”

“The dataset further models practical adversarial settings: (1) a paraphrasing attack that rewrites MGTs with the Dipper paraphraser (Krishna et al., 2023) and Polish paraphraser, and (2) a mixed-text condition where 1/4 of machine-generated sentences is randomly replaced with human-written content…”

“More specifically, paraphrasing, even by another LLM, can sharply erode detector performance; in stress tests, paraphrase attacks reduced a leading detector’s accuracy from strong to near-random levels, although retrieval-based provenance methods can partly recover signal (Krishna et al., 2023).”

“We adopt DIPPER (Krishna et al., 2023), a paraphrasing framework originally proposed for attacking AI-generated text detectors.”

“To evaluate this, we bench-marked our method against two highly efficient baselines, Charmer and DIPPER.”

“A recurring finding is that detectors that perform extremely well in-domain can degrade substantially under distribution shift (new domains, new generators) or adversarial rewriting 5,10 .”

“Furthermore, studies have reported that these methods are vulnerable to manual modifica-tions, such as paraphrasing or synonym substitution [15], [22].”

“In this subsection, we assess the robustness of our method against various countermeasures, including finetuning [2], pruning [26], quantization [26], data filtering [29], and paraphrasing [33], which are commonly employed to maliciously remove the watermark.”

“Building on the challenges posed by AIGT detection frameworks, (Krishna et al. 2023)[82] proposed DIPPER, a paraphrasing technique developed by Google that poses a huge drop in detection accuracy while maintaining the semantic equivalence of the input.”