Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

“For instance, Krishna et al. (2024); Sadasivan et al. (2023); Weber-Wulff et al. (2023) find that a simple paraphrase of machine-written text can fool widely used AI-generated content and plagiarism detectors.”

“After a careful literature survey PEGASUS was chosen for this task as it has been found to be the best approach for text paraphrasing [34].”

“Here, we call such constraints that would naturally be included in instruction and are not related to detection-evasion as task-oriented constraints.”

“By utilizing the T5-based Dipper model, we augment the original VAST dataset without changing the original meaning of the original VAST passages.”

“Empirically, paraphrasing attacks work well against post-hoc detection algorithms for LLMs [28, 30].”

“The paraphrasing attacks employed DIPPER to rephrase AI-generated texts.”

“Following prior work [19], [32]–[34], [45], [46], our previous experiments primarily used the optimal detection thresholds provided by watermark designers to ensure the best detection performance.”

“Moreover, text rephrasing models like those in (Krishna et al., 2023) are bypassing prevalent post-hoc detectors such as GPTZero (Tian, 2023), DetectGPT (Mitchell et al., 2023), and OpenAI’s proprietary detector (Kirchner et al., 2023).”

“We include other baselines to understand the impact of KiP: BART-Para , as described in subsection 3.1, and DIPPER-large (Krishna et al., 2023), which fine-tunes T5-large on the PAR3 dataset (Thai et al., 2022), a collection of aligned literary translations.”

“Following the advent of ChatGPT, scholarly interest in this domain has notably increased, as illustrated by [6-8].”

“In contrast to recent works (Sadasivan et al., 2023; Krishna et al., 2023), paraphrasing exhibits limited abilities to modify the detector’s performance.”

“We evaluate our method on two datasets commonly used in LLM watermarking: the realnewslike subset of C4 (Raffel et al., 2020) and the LFQA dataset (Krishna et al., 2023).”

“Existing approaches for detecting LLMs-generated text can be broadly categorized into three main categories [19]: watermarking [20, 21, 22, 13, 23], zero-shot methods [24, 25, 26, 27, 28, 29, 30, 31], and supervised training-based classifiers [32, 33, 34, 35, 36, 37, 38, 39].”

“The findings align with earlier research [16] [17] [18] [19] [20] [21] [22], and sharply contrast with the accuracy claims made by commercial detection tools [23] [24] [15] [22].”

“However, DIPPER was able to achieve highest semantic consistency scores on all three metrics, suggesting that it was able to generate obfuscated texts that more closely resemble the original, semantically.”

“As a final step, we have Gemini-Pro paraphrase all generated hypotheses to minimize artifacts in the data that could distinguish between hypothesis labels (Krishna et al., 2024; Sadasivan et al., 2024).”

“However, LLM red-green list watermarks are not robust against paraphrasing attacks [42, 26].”

“However, studies like [26, 32] indicate that watermarking is vulnerable to text manipulations such as paraphrasing.”

“Figure 1: Percentage of paraphrases predicted on the Semantic Text Similarity Dataset (STSBenchmark) dataset (Cer et al., 2017), binned by scores from 0 (completely dissimilar) to 5 (completely equivalent).”

“Consequently, there is a need for effective detection meth-ods (Kirchenbauer et al., 2023; Mitchell et al., 2023; Krishna et al., 2023) to address this problem (Tang et al., 2023).”

“OUTFOX The machine-generated text in the testing data is paraphrased using OUTFOX (Koike et al., 2024) or DIPPER (Krishna et al., 2024).”

“Some works (Meral et al., 2009; Krishna et al., 2023; Zhao et al., 2023; Kirchenbauer et al., 2023) explore the potential of watermarks in language models, making model-generated texts easier to detect.”

“For the human evaluation study, we select the Long-Form Question Answering (LFQA) dataset [31] to assess the robustness of the watermarking method across real-world question-answering scenarios.”

“To compare against pure semantic rewriting, we include Dipper [20], a state-of-the-art 11B model trained to paraphrase but not explicitly designed for anonymization.”

“A study from Google Research (Krishna et al. , 2023) [11] introduced a paraphrasing model called Dipper that rewrote AI text and was able to bypass many detectors nearly universally.”

“Krishna et al. (2023) reported that applying paraphrasing to LLM-generated text can bypass existing detection methods.”

“…2024), Genie (Khashabi et al., 2022), InstrSum (Ji et al., 2023), HH-RLHF Help-ful (Bai et al., 2022); document-level pairwise-classification : DIPPER (Krishna et al., 2023), and document-level bitext-mining : Europarl (Koehn, 2005), IWSLT17 (Cettolo et al., 2017), NC2016 (Maruf et al., 2019)).”

“Both Zhang et al. (2023) and Sadasivan et al. (2023) propose recursive paraphrasing pipelines, while Krishna et al. (2023) introduce DIPPER, a paraphraser specifically tuned to evade AI-generated text detection.”

“As per findings in Krishna et al. (2024), DIPPER performs at par with GPT-3.”

“Finally, the continuing challenge of generative-AI text detection is acknowledged in recent literature ( Krishna et al., 2024; Sadasivan et al., 2023), signifying a necessity for increased endeavour in this domain.”

“Evasion techniques such as prompting [24], recursive paraphrasing [25], authorship obfuscation [26], and sentence or word substitution have been developed to point out the failures in the AIGC detector tools.”

“…our paper, we evaluate seven representative detectors: RADAR (Hu et al., 2023), Detection in the Wild (Wild) (Li et al., 2024), T5Sentinel (Chen et al., 2023), Fast-DetectGPT (Bao et al., 2024), PHD (Tulchinskii et al., 2023), LogRank (Ippolito et al., 2020) 2 , and Binoculars (Hans et al., 2024).”

“Re-moving the mask bit from MC 2 M ARK leads to a noticeable performance drop when the message length becomes large.”

“Additionally, it includes over 11 adversarial attack strategies (Liang et al., 2023b;a; Wolff & Wolff, 2022; Bhat & Parthasarathy, 2020; Krishna et al., 2023; Pu et al., 2023; Gagiano et al., 2021; Guerrero et al., 2022), designed to test the robustness of detectors under challenging settings.”

“To measure detectability, we follow Kirchenbauer et al. (2023); Zhao et al. (2024) and use subsets of the news-like C4 dataset (Raffel et al., 2020) and the long-form OpenGen dataset (Krishna et al., 2023) to insert watermarks.”

“While external paraphrasing models like Dipper (Krishna et al., 2023) are available, differences in tokenizers between the LLM and these models render the operation non-differentiable.”

“Additionally, paraphrasing attacks exploit simple rephrasings to mask many stylistic signals of machine-generated text [21].”

“Early studies concentrate on identifying fully AI-generated text (Gehrmann et al., 2019; Ippolito et al., 2020; Mitchell et al., 2023), while later studies address challenges like paraphrasing and adversarial attacks (Krishna et al., 2024; He et al., 2024; Dugan et al., 2024; Wu et al., 2024).”

“Similarly, Krishna et al. (2024) acknowledge that paraphrasing is an effective way to evade detection of AI generated texts but suggest that retrieval of such paraphrased text may help improve the techniques.”

“Thus, in Algorithm 1, the h ( seed, s : k − 1 ) in the Shuf-fle function becomes h ( seed, v k − 1 ) .”

“These methods include machine translation (Altakrori et al., 2022; Keswani et al., 2016), paraphrasing (Krishna et al., 2023), or synonym replacements (Potthast et al., 2016).”

“For OpenGen and LFQA, we follow Zhao et al. (2023) to use human-written prefixes or questions as prompts and the corresponding suffixes or answers as human-written text.”

“However, Our method maintains optimal detection performance after both DIPPER and back-translation attacks.”

“For evaluation, we employ strong semantic-preserving attacks, including T5-based replacement (Raffel et al., 2020) and DIPPER paraphrasing (Krishna et al., 2023), focusing on detection performance under these attacks at low false positive rates (FPR).”

“We further evaluate MPAC, RSBH, Steal-thInk, and our proposed XM ARK on the text completion task using the OpenGen (Krishna et al., 2023) and Essays (Schuhmann, 2023) More Results on Additional LLMs.”

“In this experiment, we adopted the Dipper paraphrasing model [27], which is built on the T5-XXL [22] architecture.”

“One common strategy is the use of paraphrasing attacks [9, 10], where the generated text is automatically processed by the same model or another to paraphrase sentences in the text.”

“However, these methods tend to be biased towards the training dataset [10]–[12] and subject to adversarial attacks [13]–[18].”

“[5] discusses 'spaceinfo,' which is a tactic that involves adding spaces before commas to reduce detectability, and [6] notes how paraphrasing can help bypass detectors.”

“…et al. (2023) and other work that utilizes paraphrasing as a black-box transformation in the security and safety domain (Kirchenbauer et al., 2023; Krishna et al., 2023), we prompt a powerful large language model, GPT4-turbo, to behave as a general purpose paraphrasing model using a template…”

“…transfer Explainable formal → informal style transfer is an interpretable adversarial attack on AI-generated text detection methods, including retrieval Krishna et al. (2023) established that paraphrasing easily evades the detection of AI-generated text, and proposed a retrieval-based defense.”

“Namely, the DIPPER attack averages at most around 30% success on other three schemes, while it successfully scrubs KGW2-S UM already in around 80% of times.”

“This can be done directly using a specialized LLM (Krishna et al., 2023; Sadasivan et al., 2023), word-level substitutions (Shi et al., 2023), or translation (to another language and back) (Christ et al., 2023).”

“On the attack side, paraphrase removes token-level signals [Krishna et al., 2023], watermark stealing reverse-engineers the keyed rules from API access [Jovanović et al., 2024], Sadasivan et al. [2023] evade detectors through recursive paraphrasing, and Zhang et al. [2024] prove that a quality…”

“Meanwhile, IE and EWD show better robustness, maintaining around 90%.”

“The research paper by Krishna et al [28] examined the problem of detecting AI-generated text and how current detection algorithms are susceptible to paraphrase attacks.”

“LLMs have shown promise in rewriting MT outputs (Ki and Carpuat, 2024; Zeng et al., 2024; Xu et al., 2024), and can rewrite text according to various style specifications (Ra-heja et al., 2023; Hallinan et al., 2023; Shu et al., 2024; Krishna et al., 2024).”

“We also utilized a cleaned version of LFQA dataset from this work (Krishna et al., 2023).”

“It reduces the gap between the likelihood of the target data and the likelihood from the original model while ensuring the unlearned model remains closely aligned with the original (See Appendix B.5 for more details).”

“The statistical pattern of watermarked texts could be vulnerable to potential attacks, including lexical modification or paraphrasing [Krishna et al., 2023, Kirchenbauer et al., 2023b].”

“Krishna et al. (2024) showcased the efficacy of paraphrasing attacks towards AI generation detection models and applicable retrieval-based solutions.”

“…et al., 2019; Uchendu et al., 2020), primarily using RoBERTa (Liu, 2019), provide more accurate results than statistical methods (Li et al., 2024; Liu et al., 2023b), but lack robustness (Li et al., 2024; Koike et al., 2024; Krishna et al., 2024; Chakraborty et al., 2023; Tulchinskii et al., 2024).”

“…adjudicates one manuscript at a time, the small-sample case where the guarantee does not bite and where paraphrase defeats every scheme tested, watermarking included, dropping one detector from 70.3 to 4.6 per cent accuracy without altering meaning (Kirchenbauer et al., 2023; Krishna et al., 2023).”

“Paraphrasing is considered the most effective attack against text-or code-based watermark schemes (Krishna et al., 2023; Rastogi & Pruthi, 2024; Pang ).”

“For example, Krishna et al. (2024) enhanced machine paraphrased text detection us-ing retrieval methods, and Li et al. (2024) identified paraphrased sentences through article context.”

“In text, the most prominent attack is paraphrasing [32, 33, 6, 7], rewriting the watermarked content while preserving its meaning, using language models, translation tools, or manual edits.”

“• Humanize : Adversarial rewriting of LLM-generated essays using DIPPER [14], a discourse-aware para-phrasing model designed to conceal LLM usage.”

“…including character-and word-level perturbations (Pu et al., 2023; Wang et al., 2024; Wu et al., 2024), paraphrasing (as a defense, rather than a generation strategy) (Sadasivan et al., 2023; Krishna et al., 2024; Masrour et al., 2025), and prompt-tuning (Kumarage et al., 2023b; Shi et al., 2024).”

“Baselines Models.”

“Paraphrasing attacks are implemented using two established models: PEGA-SUS (Zhang et al., 2020) and DIPPER (Krishna et al., 2023) (configured with lexical=60 and order=40 ), following standard robustness evaluation protocols (Hou et al., 2024; Liu and Bu, 2024).”

“Computational costs further hinder scalability: DocumentMutator (based on DIPPER (Krishna et al., 2023)) took 213 seconds per attack step, rendering large-scale edits impractical.”

“Similarly, [16] introduces DIPPER, a generative model trained to paraphrase paragraphs and evade detection.”

“This limitation stems partly from the introduction of various techniques, such as recursive paraphrasing and authorship obfuscation, which can bypass these verification systems (Krishna et al., 2024).”

“We thus report the detection accuracy with a threshold by fixing the false-positive rate (FPR) at 1%, which is an evaluation stream among recent robustness studies (Krishna et al., 2023; Hans et al., 2024; Dugan et al., 2024).”

“For the robustness in recognizing the paraphrase attacks of AI-generated text, DIP-PER rephrases text generated by LLMs and bypasses several detectors to reduce the detection accuracy using semantic similarity [47].”

“Para-phrases have been shown to degrade the performance of machine-text detectors, including those based upon watermarking (Krishna et al., 2023; Sadasivan et al., 2025).”

“Currently existing perplexity based detectors are biased against non-native English writers (Liang et al., 2023), raising ethical concerns about their usage in real-world applications.”

“Third, while paraphrasing attacks have been well-studied [1, 7], more sophisticated adversarial strategies, such as embedding-based perturbations [23], demonstrate that even state-of-the-art detectors like Fast-DetectGPT can be compromised, underscoring the fragility of current systems.”

“To precisely quantify the vulnerability of AI-generated text detectors under paraphrasing attacks, Krishna et al. [17] developed a powerful paraphrasing model named DIPPER.”

“Paraphrase Attack.”

“Krishna et al. (2023) show that paraphrasing can drastically reduce detection accuracy and advocate for retrieval-based defenses.”

“RAID (Dugan et al., 2024) encompasses over 6 million text generations from 11 language models across multiple domains, incorporating adversarial techniques including paraphrase attacks via Krishna et al.’s fine-tuned T5-11B models (Krishna et al., 2023).”