Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

“Our investigation into the effectiveness of simple prompts in bypassing GPT detectors, along with recent studies on paraphrasing attacks (Krishna et al., 2023; Sadasivan et al., 2023), raises significant concerns about the reliability of current detection methods.”

“Educators have begun to use AI-powered plagiarism detectors to identify student submissions that were generated by AI, yet AI paraphrasing programs like Quillbot have been found to render AI-generated text indetectable by such tools (Krishna et al., 2023).”

“Our findings are consistent with previously published studies (Gao et al., 2022; Anderson et al., 2023; Demers, 2023; Gewirtz, 2023; Krishna et al., 2023; Pegoraro et al., 2023; van Oijen, 2023; Wang et al., 2023) and substantially differ from what some detection tools for AI-generated text claim…”

“Since for zero-shot methods, detection rates are heavily dependent on the threshold when using discriminative statistics, the AUROC metric is commonly used to measure zero-shot detector performance, which considers the range of all possible thresholds (Krishna et al., 2023).”

“The authors in [27, 17] showed that using LLMs as a paraphraser can easily evade several AI-text detection methods, even in the scenario when the original AI-text had been watermarked.”

“• We provide reliability estimates of watermarking compared to other state-of-the-art approaches, such as loss-based detection (Mitchell et al., 2023) and retrieval (Krishna et al., 2023), showing that these struggle at longer sequence lengths when attacked.”

“While it has been shown that introducing retrieval [141] can be an effective defense against paraphrasing attacks, the implementation of this defense requires maintenance by the language model API provider and remains vulnerable to recursive paraphrasing [150]. b) Adversarial Attacks: We summarise…”

“We generate x r using the paraphrasing model from Krishna et al. (2023).”

“More related to our focus on paraphrase attack, Krishna et al. (2023) propose a retrieval-based method that requires saving all previously-generated sequences, and Kirchenbauer et al. (2023b) empirically show that the robustness of the baseline algorithm is decent under relatively long generation…”

“Krishna et al. (2023) determined that before using a paraphrasing tool, DetectGPT accurately identified 70.3% of model-gener - ated sequences from GPT2-XL.”

“Most machine learning models, including large language models (LLMs), use pseudorandom number generators in one form or another to produce their outputs.”

“For DIPPER-1 the lex diversity is 60 without order diversity, and for DIPPER-2 we additionally increase the order diversity by 20.”

“These include attacks based on invisible characters and homoglyphs [16, 24, 41, 8], generative attacks such as those that prompted the model to change its output in a predictable and easily reversible way [32], and specific instances such as the emoji attack [18], and paraphrasing attacks [47, 33].”

“One potential way to counter the previous step-wise prompt defense from the attackers’ perspective is to paraphrase the instructions in the pop-ups (Krishna et al., 2024; Abdali et al., 2024).”

“However, previous work has found that watermarking can be defeated through paraphrasing input texts (Krishna et al., 2023; Sadasivan et al., 2023)”

“One is a retrieval-based approach, which involves keeping a growing record of all generated texts and checking against it for matches 12 .”

“Additionally, in another 2023 study, Krishna et al. [257] explored the vulnerability of AI-generated text detectors to paraphrase.”

“Additionally, two document-level text tampering attacks are provided: paraphrasing the context via OpenAI API or the Dipper model (Krishna et al., 2023).”

“For Dipper, We note that all methods are robust to Dipper since it does not significantly reduce the detection performance.”

“Other work has shown that it is possible to train models to transform text such that it fools text distinguishers [19, 31].”

“The experiments are divided into three scenarios: Non-attacked, DIPPER [35] attack, and OUTFOX attack, the results are presented in Table 3.”

“Ad-ditionally, Krishna et al. (2023) suggests building a database of generated text and detecting the target text by comparing its semantic similarity with all the text stored in the database.”

“All samples created were tested against seven AI text detectors which had been previously identified in literature (Chaka, 2023a; Gao et al., 2022; Krishna et al., 2023; Orenstrakh et al., 2023; Perkins, Roe, et al., 2023; Weber-Wulff et al., 2023)as being somewhat effective at successfully…”

“Prior works have investigated attacks to remove the watermark from water-marked text, e.g., by paraphrasing, swapping tokens, etc. (Kirchenbauer et al., 2023b; Krishna et al., 2023; Sadasivan et al., 2023; Zhang et al., 2023).”

“Exploring alternative training approaches such as RLHF and DPO is a promising direction for future work.”

“In addition, paraphrasing (Krishna et al., 2023; Kovatchev, 2022) and perturbation (Chen et al., 2023) approaches are widely used in NLP tasks.”

“Paraphrasing and translation are the most effective attacks: they remove the watermark most often, and do not heavily affect quality.”

“Krishna et al. (2023) build their own paraphrasing model (DIPPER) and demonstrate that input from three LLMs (GPT2, OPT-13B, GPT-3.”

“Output that is generated from conditioning on the stories can take much more from the input story, thus probably reaching a higher level of similarity, beyond what our retrieval algorithm is based on [35].”

“Such records could be retrieved to verify whether any piece of content is AI-generated [15].”

“Machine-generated text detection methods are vulnerable to authorship obfuscation attacks such as paraphrasing (Crothers et al., 2022; Krishna et al., 2023; Shi et al., 2023; Koike et al., 2023), back-translation, and change point detection.”

“5 and DIPPER (Krishna et al., 2023).”

“This has necessitated the * Corresponding author development of various methods to detect machine-generated texts through techniques such as zero-shot detection, supervised training, watermarking, and more (Mitchell et al., 2023; Wang et al., 2023b; Kirchenbauer et al., 2023a; Krishna et al., 2023).”

“Several works show that MGT detection methods are susceptible to authorship obfuscation attacks such as paraphrasing, back-translation, and human-machine collaboration mixture (Crothers et al., 2022; Krishna et al., 2023; Shi et al., 2023; Koike et al., 2023).”

“In parallel, researchers have detected text generated by language models using supervised learning and heuristics [6, 35, 55, 65, 82, 103, 113, 121], which closely resemble those in visual forensics.”

“Some works (Meral et al., 2009; Krishna et al., 2023; Zhao et al., 2023; Kirchenbauer et al., 2023) explore the potential of watermarks in language models, making model-generated texts easier to detect.”

“GPT-2 Output Detector [11] is a LLM-generated text detector based on the RoBERTa large pretrained model [19].”

“…for these approaches is their robustness to potential attacks, since a malicious user could attempt to remove the watermark with text perturbations such as editing and paraphrasing (Wang et al., 2024; Kr-ishna et al., 2023; Sadasivan et al., 2023; Kirchen-bauer et al., 2023b; Zhao et al., 2023).”

“Teachers are starting to utilize AI-powered plagiarism detectors to find student submissions that AI created; however, AI paraphrase tools, such as Quillbot, make AI-made material undetectable to these tools (Krishna et al., 2023).”

“The purification techniques for watermark and adversarial perturbations have challenged the robustness of watermark protection on model copyright in both image generation (Nie et al., 2022; Yoon et al., 2021; Chen et al., 2021a) and text generation (Krishna et al., 2023; Ren et al., 2023).”

“…engineering i.e., optimizing the prompt in LLMs to get required results (Lu et al., 2023), recursive paraphrasing i.e., paraphrasing multiple times (Krishna et al., 2024), authorship obfuscation (Macko et al., 2024), and sentence or word substitution have been developed to point out the failures…”

“Recent studies suggest that text paraphrasing significantly reduces detection accuracy (Sadasivan et al., 2023; Krishna et al., 2023).”

“A paraphrasing attack can challenge the security and reliability of LLMs and their applications (Krishna et al., 2023; Sadasivan et al., 2023).”

“Several recent works have highlighted that paraphrasing methods may be used to evade the detection of AI-generated text [18, 13, 20, 21, 43], with [18, 43] demonstrating effective watermark removal using a local LLM.”

“) but also human-level context writing skill: even with watermarks for LLM, it cannot be reliably distinguished from the generation context created by human beings (Sadasivan et al., 2023; Krishna et al., 2023).”

“For the CISR baseline, we used the open-source PyTorch implementation available at https:// github.com/nlpsoc/Style-Embeddings .”

“We utilize various techniques, including the DIPPER paraphrasing tool [23], Back-translation, and Polishing with LLMs.”

“Therefore, designing efficient frameworks for detecting synthetic data can be an interesting direction for further work.”

“Nevertheless, empirical demonstrations in (Sadasivan et al., 2023; Krishna et al., 2023) exposed the vulnerability of the watermark-based detectors with paraphrasingbased attacks, raising a genuine concern in the community about the detection of AI-generated texts.”

“These are relatively small models and there is no guarantee that our methods would work for larger models or different instruction tuning regimes.”

“As a strategy more reliable than watermarking, Krishna et al. (2023) suggest a retrieval-based approach: By storing all model outputs in a database, LLM providers can verify whether a given text was previously generated by their language model.”

“Previous works have reported such lack of compositional reasoning in both vision-and-language models (e.g., [69, 82, 85, 56, 73]) and unimodal language models (e.g., [35, 9, 88, 47]).”

“Existing studies [23, 39] show that intentional paraphrasing can effectively evade AI-generated text detection.”

“Nevertheless, AI-text detector may inherit vulnerabilities from neural network models (Szegedy et al., 2014), spurring related research (Sadasivan et al., 2023; Krishna et al., 2023) aimed at conducting paraphrasing attacks on AI detectors to mislead their predictions.”

“In this work, we propose a novel way to overcome the aforementioned challenges by proposing a method that reads part of the model parameters and computes a fingerprint for each LLM without interfering with training or exposing model parameters to the public.”

“…involve fine-tuning a pre-trained language model on a constructed dataset of both human-written and LLM-generated content; 2) Zero-shot methods [45, 63, 64, 102], which utilize intrinsic properties of typical LLMs, such as probability curve [64] and N-gram divergence [102], for self-detection;…”

“Ethical concerns arise regarding AI-generated paraphrase detection, prompting developments in specialized detection methods [88], [96].”

“Unfortunately, retrieval-based detection methods store large amounts of user-generated content and raises significant privacy concerns [147].”

“However, current zero-shot detectors are not reliable enough when dealing with cleverly revised text, such as paraphrased text (Sadasivan et al., 2023; Krishna et al., 2023; Wu et al., 2024).”

“Some works (Kirchenbauer et al., 2023b; Liu et al., 2023) focus on improving the detection robustness to paraphrasing attacks (Krishna et al., 2023) or at low-entropy environment (Lu et al., 2024).”

“Previous studies (Krishna et al., 2024; Sadasivan et al., 2023) show that paraphrasing attacks can effectively evade detection by methods like watermarking, GPTZero, DetectGPT, and OpenAI’s text classifier.”

“Furthermore, in accordance with the recommendations put forth by (Krishna et al., 2023; Yang et al., 2023), we prioritize maintaining a high True Positive Rate (TPR) while minimizing the False Positive Rate (FPR).”

“Data for Wikipedia and Reddit was taken from [Krishna et al., 2023]; as for StackExchange, we assembled the dataset ourselves following the same scheme as Krishna et al. [2023].”

“We hope that our work can help the community better understand the CTWL issue and inspire more outstanding research in the future.”

“To validate the performance of our method, we conduct extensive experiments on multiple benchmark datasets (Open-Gen [35], C4 news dataset [55], Essays dataset [61]) with multiple large language models (LLaMA-2-7B [64], Guanaco-7B [17], and Falcon-7B [3]).”

“Our investigation into the effectiveness of simple prompts in bypassing GPT detectors, along with recent studies on paraphrasing attacks (Krishna et al., 2023; Sadasivan et al., 2023), raises significant concerns about the reliability of current detection methods.”

“To evaluate the robustness of AIGT detectors under different misclassification cost variations, we conducted experiments following the 1% false positive rate (FPR) setting proposed in (Hans et al., 2024; Krishna et al., 2024; Soto et al., 2024).”

“Additionally, Krishna et al. (2024) developed a detector that uses information retrieval to store LLM outputs in a database and search for semantically similar content to identify LLM-generated text, though this method raises privacy concerns regarding the storage of user conversations.”

“This method was recently demonstrated for text by Krishna et al. (2023). plagiarism detector is essentially an information retrieval (IR) device: the companies at the fore-front of FM content generation also have huge expertise in this area, and would be very well placed to provide detectors of…”

“It is also worth noting that identifying AI-generation style is difficult, and using it to poison a model presents significant challenges [25, 12, 32, 31].”

“5 test dataset (Krishna et al. 2023).”

“…(ii) the false positive rate (FPR) of the watermark, i.e. the ratio of human-written texts falsely flagged as watermarked, and (iii) the scrubbing robustness of the watermark, i.e., the TPR after each model response is paraphrased using a state-of-the-art paraphraser (Krishna et al., 2023).”

“…tasks and metrics to test the quality of watermarks (Piet et al., 2023), some with a special focus on text quality (Tang et al., 2023; Tu et al., 2023; Ajith et al., 2023) or robustness (Krishna et al., 2023; Sadasivan et al., 2023; Shi et al., 2023; Kirchenbauer et al., 2023b; Zhang et al., 2023).”

“Thus, these studies assume that authorship remains the same after paraphrasing .”

“…al., 2023a; Takezawa et al., 2023; Christ et al., 2023) and post-hoc watermarking as He et al. (2022a) and Venugopal et al. (2011) and ours, retrieval-based detection (Krishna et al., 2023), and linguistic steganography (Fang et al., 2017; Dai & Cai, 2019; Ziegler et al., 2019; Ueoka et al., 2021).”

“A recent paper (Krishna et al., 2023) Another paper (Sadasivan et al., 2023) also uses the DIPPER paraphrasing technique but a slightly modified version in which they use parallel paraphrasing of multiple sentences.”

“In a paper Kalpesh Krishna et al. [20] created a substantial amount of AI-generated text samples using a number of cutting-edge language models, such as GPT-3 and T5.”

“This has necessitated the * Corresponding author development of various methods to detect machine-generated texts through techniques such as zero-shot detection, supervised training, watermarking, and more (Mitchell et al., 2023; Wang et al., 2023b; Kirchenbauer et al., 2023a; Krishna et al., 2023).”

“To address this, we recommend moving beyond binary classification frameworks and adopting tiered or probabilistic labeling schemes that reflect varying degrees of AI involvement.”

“Unlike prior attacks (Krishna et al., 2023; Kirchenbauer et al., 2023b), we first attempt to decipher the tokens in the green list and then incorporate that knowledge in existing paraphrasing attacks.”

“M ); (ii) are more generalizable to expert-and LLM-edited texts, which is indicated by lower AUROC δ -scores (e.g., up to 10% for Log-Rank, DetectLLM LLR/NPR, DetectGPT and up to 20% for AIGC MPU, MAGE, and OpenAI RoBERTa-base/large); and (iii) perform better with a larger backbone model in the H…”

“Our evaluation employs two datasets: 1) OpenGen [11] for text completion task, consisting of 3K two-sentence samples from WikiText-103 [16], with the first sentence as the prompt and the second as the human completion; 2) LFQA [11] for long-form question answering task, consisting of 3K…”

“We paraphrase responses by two paraphrasing models: Pegasus (Zhang et al., 2019) and DIPPER (Krishna et al., 2023).”

“NLP functions as the internal working mechanics for online writing programs, such as Grammarly and Quillbot (Krishna et al., 2023).”