Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

“Although different guidance detectors may yield slightly varying degrees of transferability depending on the deployed detector, all our adversarially paraphrased outputs, agnostic The first row corresponds to simple (non-adversarial) paraphrasing baseline [15].”

“Our findings are consistent with previously published studies (Gao et al., 2022; Anderson et al., 2023; Demers, 2023; Gewirtz, 2023; Krishna et al., 2023; Pegoraro et al., 2023; van Oijen, 2023; Wang et al., 2023) and substantially differ from what some detection tools for AI-generated text claim…”

“5 or Dipper, where MorphMark exp achieves a substantially higher TPR@1%.”

“A study from Google Research (Krishna et al., 2023) found that a paraphrasing text-to-text model (a variant of T5) was able to effectively rewrite AI-generated text in a way that could preserve the meaning of the original text but largely evade AI detection algorithms.”

“A high ASR of Pegasus-parahrase cannot be solely due to changed language of the texts after obfuscation, since DIPPER changed the language for almost twice as many texts but achieves only about a half ASR.”

“Despite this strategy’s potential, recent studies indicate that these watermarks can be readily eliminated through the use of automatic paraphrasing techniques (Sadasivan et al., 2023; Krishna et al., 2023).”

“Besides, the text rephrasing model in (Krishna et al., 2023) bypassing prevalent post-hoc detectors like GPTZero (Tian, 2023), DetectGPT (Mitchell et al., 2023), and OpenAI’s proprietary detector (Kirchner et al., 2023).”

“On a related note, Krishna [17] proposes evaluating the efficacy of text detectors by introducing a parameter paraphrase generation model or Discourse Paraphraser (DIPPER), capable of paraphrasing entire paragraphs while considering contextual cues, regulating linguistic variety, and managing…”

“Furthermore, these detectors can be fooled easily with “paraphrasing attacks” even with in-domain samples (Sadasivan et al., 2023; Krishna et al., 2023).”

“Notably, the closest competitors to our method are DIPPER (Krishna et al., 2024) and GPT Paraphraser, which are model-based paraphrasing attacks.”

“Prior work has shown that AUC is not necessarily reflective of how models perform at very low FPR values (Yang et al., 2023; Krishna et al., 2024; Tufts et al.).”

“In addition, some papers also illustrate the ways to avoid LLM detection (Krishna et al., 2023; Koike et al., 2023), which makes this game harder.”

“Min-K% Prob (Shi et al. 2023).”

“These attacks are performed using DIPPER (Krishna et al., 2023), a fine-tuned language model for watermark evasion with controllable lexical and order diversity parameters.”

“But, other preemptive strategies such as the storage and retrieval of generated content are also helpful (Krishna et al., 2023).”

“This involved partitioning the vocabulary and only sampling tokens from one region when selecting Krishna et al. [107] devised a retrieval-based watermarking method that stores the generated contents within a database.”

“Further efforts have focused on handling adversarial attacks (Crothers et al., 2022; Krishna et al., 2023; Koike et al., 2024; Sadasivan et al., 2025), short texts (Tian et al., 2024), out-of-distribution texts Guo et al. (2024b), biases against non-native English writers (Liang et al., 2023), as…”

“The results show that frequency-based prefix weighting, which assigns higher weights to more easily learned p -rules, achieves better wa-Table termark removal while maintaining an equal total weight across prefixes.”

“However, token-level water-marks can still be vulnerable to paraphrasing, as rephrasing may disrupt the green and red token lists without altering the overall semantic (Krishna et al., 2024).”

“2 } portion of the text to-kens through token replacement attack (Kirchen-20594 (Krishna et al., 2023).”

“Existing research [20, 36] has pointed out that the previous methods experience a reduction in performance in complex scenarios where the text to be detected is subjected to interference.”

“While one approach is to record (Krishna et al., 2023) or watermark all generated text (Kirchenbauer et al., 2023 1.”

“However, score-based methods often rely on prior knowledge about a specific generator and/or semantic domain, and known traces may be easy to remove, e.g., by paraphrasing the text (Krishna et al., 2023).”

“For instance, in experiments conducted by [20], both the AI-generated and human-written texts were limited to approximately 300 tokens.”

“An adversary can paraphrase LLM-generated reviews (Krishna et al., 2023) or design prompts to guide LLMs in generating human-like text to bypass detectors (Wu et al., 2024), while these reviews still rely on LLM content that lacks a deep understanding of the paper.”

“Additionally, it has been shown that paraphrasing can successfully evade detectors [60, 61].”

“While LLMs can prove beneficial (Ngo, 2023), there is growing concern about their potential misuse (Sullivan et al., 2023; Yan et al., 2024; Li et al., 2023; Sebastian, 2023).”

“Therefore, exploring approaches to identify paraphrased content within articles, such as Krishna et al. (2024), represents another area for further work.”

“…built for paraphrasing, specifically built to paraphrase text automatically and evade LLM-generated text detectors; an example of this is DIPPER [19], which could paraphrase long paragraphs and control output diversity, and (ii) AI chatbots that receive paraphrasing prompts to produces…”

“Our attacker uses paraphraser models with at most 7b parameters, which is less than the 11b DIPPER model (Krishna et al., 2023) currently used to test robustness.”

“For context-aware para-phrasing with Dipper, we use the default setting with lexical diversity set at 80 and order diversity set at 60.”

“Krishna et al. [20] demonstrated that simple paraphrasing using models like DIP-PER can effectively remove watermarks while preserving semantic content.”

“In our robustness testing, we evaluate two configurations of the DIPPER (Krishna et al., 2023) model: DIPPER-1 with lexical diversity L=40, order diversity O=40, and DIPPER-2 with L=40, O=100.”

“To simulate this, we run C HAMELEON (FT+A UGS ) in interleaved mode on 1000 prompts to produce text and an image, and we model attempts to conceal the use of M by randomly changing text tokens (a proxy for paraphrasing [52]).”

“We follow the implementation of (Wang et al., 2024; Bao et al., 2024) to generate machine texts for XSum and SQuAD while adopting the approach of Dipper (Krishna et al., 2023) for LongQA.”

“In paraphrasing attacks, groups of words generated by target LLMs are replaced with semantically similar ones via specific LLMs [83], [84], word-level substitutions [85], or translation [86].”

“Some studies also explore methods to control para-phrase generation by focusing on aspects such as syntactic and lexical diversity (Li et al., 2019; Goyal and Durrett, 2020; Huang and Chang, 2021; Bandel et al., 2022; Krishna et al., 2023; Yang et al., 2022).”

“Krishna et al. (2023) Paraphrasing reduces detection accuracy significantly across all language models, detectors, and tasks, regardless of diversity control codes.”

“For zero-shot methods [43, 38, 27–29, 44, 45, 34, 46, 14, 47–49] in the black-box detection settings, they usually rely on a surrogate model for scoring.”

“However, post-hoc detection methods are inferior to watermarking methods in terms of detection accuracy (Krishna et al., 2023).”

“This involved partitioning the vocabulary and only sampling tokens from one region when selecting Krishna et al. [107] devised a retrieval-based watermarking method that stores the generated contents within a database.”

“Paraphrasing attacks (Krishna et al., 2023; Sadasivan et al., 2023) or basic syn-onym replacements (Wang et al., 2024a) are effec-3763 tive.”

“Another approach [27] relies on retrieving semantically-similar generations from a huge database with language model historical outputs.”

“Medium : We leverage DIPPER-11B (Krishna et al., 2023), a SoTA paraphrasing model to generate more complex textual variations of the original question (See Table 7).”

“…methods are often not even robust against common post-processing, e.g., JPEG compression removes the image watermark inserted by non-learning-based image watermarking meth-ods [46] and paraphrasing removes the text watermark inserted by non-learning-based text watermarking meth-ods [49, 70].”

“A seminal paraphrasing model called Dipper initially had a high bypass rate against our detector (Krishna et al., 2023).”

“In the paraphrasing attack, we use the Dipper (Krishna et al., 2023) and PEGASUS (Zhang et al., 2020) models to paraphrase the watermarked text, with ϵ representing the paraphrasing strength.”

“…et al. [44] show that despite efforts to scrub sensitive information from models, different attacks can still locate such information in LMs. Black-box attacks assume no access to model internals and aim to retrieve information through conversation [26], designing prompts [36] or paraphrasing [31].”

“Proposed solutions involve semantic retrieval to counter paraphrase attack (Krishna et al., 2023), but they rely on text generation APIs like the OpenAI API, which limits their practical applicability when evaluating independent detection mechanisms.”

“We fixed the perturbation rate at 10% across all experiments, which is less than the rate set in Shi et al. (2024) and Krishna et al. (2023).”

“In these methods, the provider of the AI generator keeps a private log of content it generates (see Krishna et al., 2023, for the original proposal).”

“However, these methods tend to be biased towards the training dataset [8, 18, 29] and subject to adversarial attacks [5, 13, 22].”

“[13] For testing the first hypothesis, independent samples t-test was performed on the post-test data involving the control and experimental groups.”

“Dipper is a model with 11B parameters fine-tuned for paraphrasing, causing a significant modification of the text.”

“Although retrieval [6] can help mitigate this problem, it still poses significant challenges, such as high-costing and lacking explanation.”

“Krishna et al. (2023) train a generative model (DIPPER) to paraphrase paragraphs to evade detection.”

“Perhaps unsurprisingly, paraphrasing attacks (consisting in the replacement of some words in the text with synonyms) have been shown to successfully bypass several AI detectors [29], and paraphrasing tools such as Quillbot or GPT-Minus1 are widespread.”

“It also includes adversarially paraphrased samples generated using DIPPER [77] and OUTFOX attack methods.”

“Recent studies show that paraphrasing (Sadasivan et al., 2023; Krishna et al., 2023) and word-level perturbations (Peng et al., 2023; Liang et al., 2023) can effectively evade current AI-text detectors by altering lexical and syntactic features.”

“No-tably, under the Unigram watermarking strategy, watermark General capability.”

“Paraphrase attacks [23] focus on rewriting text while maintaining its original meaning to evade detection.”

“This can involve various adversarial techniques, such as introducing noise, paraphrasing, altering sentence structures, or using syn-onyms to obscure the original text [84,85].”

“The reliability of synthetic text recognition remains variable, although it has improved significantly [57], [58], [59], [60], [61].”

“Particularly in recent years, there is a trend toward practical applications, emphasizing high true positive rates at low false positive rates, suggesting that at least an AUC in the late 0.9s would be necessary [31, 8].”

“Krishna et al. [126] used DIPPER, an 11-billion-parameter paraphraser, to reword AI-generated text, reducing detection accuracy for various tools (GPTZero, De-tectGPT, OpenAI’s classifier) from as high as 70% to under 5%.”

“…CondaQA edits that change how negation is expressed, but keep the meaning intact, builds on work in prompting LLMs for paraphrasing (Cegin et al., 2023; Witteveen and Andrews, 2019; Pehli-vanoglu et al., 2024) and controllable paraphrasing (Hosking and Lapata, 2021; Krishna et al., 2023).”

“Third, most existing datasets lack practical adversarial scenarios, such as paraphrasing [41, 68], which are common in practice and essential for evaluating the robustness of detection systems.”

“For paraphrasing attacks, we employ DIPPER with hyperparameters set to a lexical diversity of 60 and a syntactic diversity of 60.”

“We implemented the input rephrasing attack (Patil et al., 2023; Krishna et al., 2024) to generate multiple attack templates.”

“For instance, Dipper(Krishna et al., 2023) is a widely used paraphrasing attack to evaluate the robustness of LLM watermarkers (Kirchen-bauer et al., 2023a; Zhao et al., 2023) against watermark removal attacks; the watermark stealing attack(Jovanovi´c et al., 2024) is proposed to identify…”

“Paraphrasing tools are effective in evading detection (Sadasivan et al., 2023b; Krishna et al., 2024).”

“However, despite the numerous studies targeting MGT detection, Krishna et al. [57] highlight that paraphrased text escapes the existent detectors, including watermarking, DetectGPT, or GPTZero, with an important drop in performance.”

“As detection of text generated by Large Language Models (LLMs) seemingly approaches infeasibility (Jawahar et al., 2020; Krishna et al., 2023; Sadasivan et al., 2023; Chakraborty et al., 2023), the ability to intentionally plant a watermark in LLM-generated text becomes the most viable approach to…”

“Many of the existing benchmarks employ artificial, controlled environments, such as directly Preprint Mitchell et al. (2023); Guo et al. (2023); Krishna et al. (2023).”

“Krishna et al. (2023) showed rephrase can remove watermark.”

“PaCoST employs a paired t-test that compares model confidence on original Table 1: P-values for detecting test-set contamination for different methods.”

“The robustness of a watermark enhances its resistance to various types of modifications, such as paraphrasing (Krishna et al., 2023).”