CoChat
Sign Up
PreprintCited by 51Topic ModelingAcademic integrity and plagiarismNatural Language Processing Techniques

Do Language Models Plagiarize?

Open access
Lee, Jooyoung, Le, Thai, Chen, Jinghui, Lee, Dongwon·2022·arXiv
DOI 10.48550/arxiv.2203.07618arXiv 2203.07618
Open this paper in CoChat — free accountOpen PDF

Abstract

Past literature has illustrated that language models (LMs) often memorize parts of training instances and reproduce them in natural language generation (NLG) processes. However, it is unclear to what extent LMs "reuse" a training corpus. For instance, models can generate paraphrased sentences that are contextually similar to training samples. In this work, therefore, we study three types of plagiarism (i.e., verbatim, paraphrase, and idea) among GPT-2 generated texts, in comparison to its training data, and further analyze the plagiarism patterns of fine-tuned LMs with domain-specific corpora which are extensively used in practice. Our results suggest that (1) three types of plagiarism widely exist in LMs beyond memorization, (2) both size and decoding methods of LMs are strongly associated with the degrees of plagiarism they exhibit, and (3) fine-tuned LMs' plagiarism patterns vary based on their corpus similarity and homogeneity. Given that a majority of LMs' training data is scraped from the Web without informing content owners, their reiteration of words, phrases, and even core ideas from training sets into generated texts has ethical implications. Their patterns are likely to exacerbate as both the size of LMs and their training data increase, raising concerns about indiscriminately pursuing larger models with larger training corpora. Plagiarized content can also contain individuals' personal and sensitive information. These findings overall cast doubt on the practicality of current LMs in mission-critical writing tasks and urge more discussions around the observed phenomena. Data and source code are available at https://github.com/Brit7777/LM-plagiarism.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2203.07618

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Foundation model monocultures have been conjectured to lead to correlated failures and cultural homogenization.

    “foundation model monocultures have been conjectured to lead to correlated failures (Bommasani et al., 2022) and cultural homogenization (Lee et al., 2022;Padmakumar & He, 2023).”

    From On the Societal Impact of Open Foundation Models (2024) · cites this paper as Lee et al., 2022

    Lee et al. is cited for cultural homogenization, not for correlated failures. The Abstract discusses plagiarism in generated text, and §6.2 discusses similarity within fine-tuning datasets, but neither addresses cultural homogenization caused by foundation model monocultures. Searches for cultural homogenization, monocultures, and cultural diversity found no passage on that claim. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 56

  1. David H Ackley, Geofrey E Hinton, and Terrence J Sejnowski. 1985. A learning algorithm for Boltzmann machines. Cognitive science 9, 1 (1985), 147-169.
  2. Basant Agarwal, Heri Ramampiaro, Helge Langseth, and Massimiliano Ruocco. 2018. A deep network model for paraphrase detection in short text messages. Information Processing & Management 54, 6 (2018), 922-937.
  3. Alim Al Ayub Ahmed, Ayman Aljabouh, Praveen Kumar Donepudi, and Myung Suh Choi. 2021. Detecting Fake News using Machine Learning: A Sys- tematic Literature Review. arXiv preprint arXiv:2102.04458 (2021). · arXiv
  4. Alaa Saleh Altheneyan and Mohamed El Bachir Menai. 2020. Automatic plagia- rism detection in obfuscated text. Pattern Analysis and Applications 23, 4 (2020), 1627-1650.
  5. Salha Alzahrani. 2015. Arabic plagiarism detection using word correlation in N-Grams with K-overlapping approach. In Proceedings of the Workshops at the 7th Forum for Information Retrieval Evaluation (FIRE). 123-125.
  6. Alberto Barrón-Cedeño, Marta Vila, M Antònia Martí, and Paolo Rosso. 2013. Plagiarism meets paraphrasing: Insights for the next generation in automatic plagiarism detection. Computational Linguistics 39, 4 (2013), 917-947.
  7. Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R Varshney. 2020. Mirostat: A neural text decoding algorithm that directly controls perplexity. arXiv preprint arXiv:2007.14966 (2020). · arXiv
  8. Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. 2022. What Does it Mean for a Language Model to Preserve Privacy? arXiv preprint arXiv:2202.05520 (2022). · arXiv
  9. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying Memorization Across Neural Language Models. arXiv preprint arXiv:2202.07646 (2022). · arXiv
  10. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19). 267-284.
  11. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21). 2633-2650.
  12. Roger Clarke. 2006. Plagiarism by academics: More complex than it seems. Journal of the Association for Information Systems 7, 1 (2006), 5.
  13. Georgina Cosma and Mike Joy. 2008. Towards a defnition of source-code plagia- rism. IEEE Transactions on Education 51, 2 (2008), 195-200.
  14. Alexandra DeLucia, Aaron Mueller, Xiang Lisa Li, and João Sedoc. 2020. Decoding methods for neural narrative generation. arXiv preprint arXiv:2010.07375 (2020). · arXiv
  15. Liming Deng, Jie Wang, Hangming Liang, Hui Chen, Zhiqiang Xie, Bojin Zhuang, Shaojun Wang, and Jing Xiao. 2020. An iterative polishing framework based on quality aware masked language model for Chinese poetry generation. In Proceedings of the AAAI conference on artifcial intelligence, Vol. 34. 7643-7650.
  16. Ish Kumar Dhammi and Rehan Ul Haq. 2016. What is plagiarism and how to avoid it? Indian journal of orthopaedics 50, 6 (2016), 581.
  17. Julianne East. 2010. Judging plagiarism: a problem of morality and convention. Higher Education 59, 1 (2010), 69-83.
  18. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833 (2018). · arXiv
  19. R. Stuart Geiger. 2019. ArXiV Archive: A tidy and complete archive of metadata for papers on arxiv.org, 1993-2019. https://doi.org/10.5281/zenodo.2533436 · DOI
  20. Deepa Gupta, K Vani, and LM Leema. 2016. Plagiarism detection in text docu- ments using sentence bounded stop word n-grams. Journal of Engineering Science and Technology 11, 10 (2016), 1403-1420.
  21. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 (2019). · arXiv
  22. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2019. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650 (2019). · arXiv
  23. Nikhil Kandpal, Eric Wallace, and Colin Rafel. 2022. Deduplicating training data mitigates privacy risks in language models. arXiv preprint arXiv:2202.06539 (2022). · arXiv
  24. Adam Kilgarrif and Tony Rose. 1998. Measures for corpus similarity and homo- geneity. In Proceedings of the Third Conference on Empirical Methods for Natural Language Processing. 46-52.
  25. Robin Küppers and Stefan Conrad. 2012. A Set-Based Approach to Plagiarism Detection.. In CLEF (Online Working Notes/Labs/Workshop).
  26. Laida Kushnareva, Daniil Cherniavskii, Vladislav Mikhailov, Ekaterina Arte- mova, Serguei Barannikov, Alexander Bernstein, Irina Piontkovskaya, Dmitri Piontkovski, and Evgeny Burnaev. 2021. Artifcial text detection via examining the topology of attention maps. arXiv preprint arXiv:2109.04825 (2021). · arXiv
  27. Jieh-Sheng Lee and Jieh Hsiang. 2020. Patent claim generation by fne-tuning OpenAI GPT-2. World Patent Information 62 (2020), 101983.
  28. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499 (2021). · arXiv
  29. Klas Leino and Matt Fredrikson. 2020. Stolen Memories: Leveraging Model Mem- orization for Calibrated {White-Box} Membership Inference. In 29th USENIX Security Symposium (USENIX Security 20). 1605-1622.
  30. Sharon Levy, Michael Saxon, and William Yang Wang. 2021. Investigating Memorization of Conspiracy Theories in Text Generation. arXiv preprint arXiv:2101.00379 (2021). · arXiv
  31. Zehui Lin, Xiao Pan, Mingxuan Wang, Xipeng Qiu, Jiangtao Feng, Hao Zhou, and Lei Li. 2020. Pre-training multilingual neural machine translation by leveraging alignment information. arXiv preprint arXiv:2010.03142 (2020). · arXiv
  32. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019). · arXiv
  33. R Thomas McCoy, Paul Smolensky, Tal Linzen, Jianfeng Gao, and Asli Celikyilmaz. 2021. How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN. arXiv preprint arXiv:2111.09509 (2021). · arXiv
  34. Derek Miller. 2019. Leveraging BERT for extractive text summarization on lectures. arXiv preprint arXiv:1906.04165 (2019). · arXiv
  35. Fatemehsadat Mireshghallah, Huseyin A Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, and Robert Sim. 2021. Privacy regularization: Joint privacy-utility optimization in language models. arXiv preprint arXiv:2103.07567 (2021). · arXiv
  36. John X Morris, Eli Lifand, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020. Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp. arXiv preprint arXiv:2005.05909 (2020). · arXiv
  37. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook FAIR's WMT19 news translation task submission. arXiv preprint arXiv:1907.06616 (2019). · arXiv
  38. Chris Park. 2003. In other (people's) words: Plagiarism by university students- literature and lessons. Assessment & evaluation in higher education 28, 5 (2003), 471-488.
  39. John Pavlopoulos, Jefrey Sorensen, Lucas Dixon, Nithum Thain, and Ion An- droutsopoulos. 2020. Toxicity detection: Does context really matter? arXiv preprint arXiv:2006.00998 (2020). · arXiv
  40. Diane Pecorari. 2008. Academic writing and plagiarism: A linguistic analysis. Bloomsbury Publishing.
  41. Robin L Plackett. 1983. Karl Pearson and the chi-squared test. International statistical review/revue internationale de statistique (1983), 59-72.
  42. Alec Radford, Jefrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  43. Colin Rafel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unifed text-to-text transformer. J. Mach. Learn. Res. 21, 140 (2020), 1-67.
  44. Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. Okapi at TREC-3. Nist Special Publication Sp 109 (1995), 109.
  45. Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. 2020. {Updates-Leak}: Data Set Inference and Reconstruction Attacks in Online Learning. In 29th USENIX Security Symposium (USENIX Security 20). 1291-1308.
  46. Miguel A Sanchez-Perez, Alexander Gelbukh, and Grigori Sidorov. 2015. Adaptive algorithm for plagiarism detection: The best-performing approach at PAN 2014 text alignment competition. In International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, 402-413.
  47. Zhonghao Sheng, Kaitao Song, Xu Tan, Yi Ren, Wei Ye, Shikun Zhang, and Tao Qin. 2020. Songmass: Automatic song writing with pre-training and alignment constraint. arXiv preprint arXiv:2012.05168 (2020). · arXiv
  48. Ilya Sochenkov, Denis Zubarev, Ilya Tikhomirov, Ivan Smirnov, Artem Shelmanov, Roman Suvorov, and Gennady Osipov. 2016. Exactus like: Plagiarism detection in scientifc texts. In European conference on information retrieval. Springer, 837-840.
  49. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. 2020. Authorship attribu- tion for neural text generation. In Conf. on Empirical Methods in Natural Language Processing (EMNLP).
  50. Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. Tur- ingBench: A Benchmark Environment for Turing Test in the Age of Neural Text Generation. In Findings of Conf. on Empirical Methods in Natural Language Processing (EMNLP-Findings).
  51. K Vani and Deepa Gupta. 2017. Detection of idea plagiarism using syntax- semantic concept extractions with genetic algorithm. Expert Systems with Appli- cations 73 (2017), 11-26.
  52. Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Darrin Eide, Kathryn Funk, Rodney Kinney, Ziyang Liu, William Merrill, et al. 2020. Cord-19: The covid-19 open research dataset. ArXiv (2020).
  53. Debora Weber-Wulf. 2019. Plagiarism detectors are a crutch, and a problem. Nature 567, 7749 (2019), 435-436.
  54. Max Wolf and Stuart Wolf. 2020. Attacking neural text detectors. arXiv preprint arXiv:2002.11768 (2020). · arXiv
  55. Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Victor Rühle, Andrew Paverd, Olga Ohrimenko, Boris Köpf, and Marc Brockschmidt. 2020. Analyzing information leakage of updates to natural language models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. 363-375.
  56. Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. 2021. Counterfactual Memorization in Neural Language Models. arXiv preprint arXiv:2112.12938 (2021). · arXiv

Cited by

Sorted by citations

Authors
Lee, J., Le, T., Chen, J., & Lee, D.
Affiliations
  • Pennsylvania State University (US)
  • University of Mississippi (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Pages
3637-3647
DOI
10.48550/arxiv.2203.07618
arXiv
2203.07618
OpenAlex
W4320458011
Cited by
51
References
0
Citations by year
2026: 9 · 2025: 15 · 2024: 16 · 2023: 9 · 2022: 2
FWCI
8.70
Type
conference paper
Language
en
Open access
Licence
Locations
  • International World Wide Web Conference (WWW) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (1934782, 2114824)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
GPT-2, fine-tuned language models, model size, decoding algorithms, natural language generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review