CoChat
Sign Up
PreprintCited by 37Academic integrity and plagiarismSoftware Engineering ResearchArtificial Intelligence in Education

Fooling MOSS Detection with Pretrained Language Models

Open access
Biderman, Stella, Raff, Edward·2022·arXiv
DOI 10.48550/arxiv.2201.07406arXiv 2201.07406
Open this paper in CoChat — free account

Abstract

As artificial intelligence (AI) technologies become increasingly powerful and prominent in society, their misuse is a growing concern. In educational settings, AI technologies could be used by students to cheat on assignments and exams. In this paper we explore whether transformers can be used to solve introductory level programming assignments while bypassing commonly used AI tools to detect similarities between pieces of software. We find that a student using GPT-J [Wang and Komatsuzaki, 2021] can complete introductory level programming assignments without triggering suspicion from MOSS [Aiken, 2000], a widely used software similarity and plagiarism detection tool. This holds despite the fact that GPT-J was not trained on the problems in question and is not provided with any examples to work from. We further find that the code written by GPT-J is diverse in structure, lacking any particular tells that future plagiarism detection techniques may use to try to identify algorithmically generated code. We conclude with a discussion of the ethical and educational implications of large language models and directions for future research.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2201.07406

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 60

  1. [1][1] Saida Affouneh, Soheil Salha, and Zuheir N Khlaif. 2020. Designing quality e-learning environments for emergency remote teaching in coronavirus crisis. Interdisciplinary Journal of Virtual Learning in Medical Sciences 11, 2 (2020), 135137. -
  2. [3][3] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 (2021). - · arXiv
  3. [5][5] Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2021. The values encoded in machine learning research. arXiv preprint arXiv:2106.15590 (2021). - · arXiv
  4. [6][6] Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. 2022. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745 (2022). - · arXiv
  5. [7][7] Kevin W Bowyer and Lawrence O Hall. 1999. Experience using MOSS to detect cheating on programming assignments. In FIE'99 Frontiers in Education. 29th Annual Frontiers in Education Conference. Designing the Future of Science and Engineering Education. Conference Proceedings (IEEE Cat. No. 99CH37011 , Vol. 3. IEEE, 13B3-18. -
  6. [8][8] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda - Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877-1901. -
  7. [9][9] Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th { USENIX } Security Symposium ( { USENIX } Security 19) . 267284. -
  8. [10][10] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel HerbertVoss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In 30th { USENIX } Security Symposium ( { USENIX } Security 21) . 26332650. -
  9. [11][11] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea. Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, David W. Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William H. Guss, Alex Nichol, Igor Babuschkin, S. Arun Balaji, Shantanu Jain, Andrew Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew M. Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). - · arXiv
  10. [12][12] Xiang Chen, Xin Xie, Ningyu Zhang, Jiahuan Yan, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2021. Adaprompt: Adaptive prompt-based finetuning for relation extraction. arXiv preprint arXiv:2104.07650 (2021). - · arXiv
  11. [13][13] Miranda Chong, Lucia Specia, and Ruslan Mitkov. 2010. Using natural language processing for automatic detection of plagiarism. In Proceedings of the 4th International Plagiarism Conference (IPC-2010) . -
  12. [14][14] Michelle Navarre Cleary. 2017. Top 10 reasons students plagiarize & what teachers can do about it (with apologies to David Letterman). Phi Delta Kappan 99, 4 (2017), 66-71. -
  13. [15][15] Katherine Crowson, Stella Biderman, Daniel Kornis, Dashiell Stander, Eric Hallahan, Louis Castricato, and Edward Raff. 2022. VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance. arXiv preprint arXiv:2204.08583 (2022). - · arXiv
  14. [16][16] Tomasz Darmetko. 2021. Fake or not? Generating adversarial examples from language models. Doctoral thesis submitted to Maastricht University. -
  15. [17][17] Phillip Dawson, Wendy Sutherland-Smith, and Mark Ricksen. 2020. Can software improve marker accuracy at detecting contract cheating? A pilot study of the Turnitin authorship investigate alpha. Assessment & Evaluation in higher education 45, 4 (2020), 473-482. -
  16. [18][18] Breanna Devore-McDonald and Emery D Berger. 2020. Mossad: defeating software plagiarism detection. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1-28. -
  17. [19][19] Iddo Drori, Sunny Tran, Roman Wang, Newman Cheng, Kevin Liu, Leonard Tang, Elizabeth Ke, Nikhil Singh, Taylor L Patti, Jayson Lynch, et al. 2021. A Neural Network Solves and Generates Mathematics Problems by Program Synthesis: Calculus, Differential Equations, Linear Algebra, and More. arXiv preprint arXiv:2112.15594 (2021). - · arXiv
  18. [20][20] Iddo Drori and Nakul Verma. 2021. Solving Linear Algebra by Program Synthesis. arXiv preprint arXiv:2111.08171 (2021). - · arXiv
  19. [21][21] Esra Eret and Tuba Gokmenoglu. 2010. Plagiarism in higher education: A case study with prospective academicians. Procedia-Social and Behavioral Sciences 2, 2 (2010), 3303-3307. -
  20. [22][22] Jinan AW Faidhi and Stuart K Robinson. 1987. An empirical approach for detecting program similarity and plagiarism within a university programming environment. Computers & Education 11, 1 (1987), 11-19. -
  21. [23][23] Vitaly Feldman. 2020. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing . 954-959. -
  22. [24][24] Tomáš Folt` ynek, Terry Ruas, Philipp Scharpf, Norman Meuschke, Moritz Schubotz, William Grosky, and Bela Gipp. 2020. Detecting machine-obfuscated plagiarism. In International Conference on Information . Springer, 816-827. -
  23. [25][25] Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. InCoder: A Generative Model for Code Infilling and Synthesis. arXiv preprint arXiv:2204.05999 (2022). - · arXiv
  24. [26][26] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The Pile: An 800GB Dataset of Diverse Text for Language Modeling. arXiv preprint arXiv:2101.00027 (2020). - · arXiv
  25. [27][27] Seungju Han, Beomsu Kim, Jin Yong Yoo, Seokjun Seo, Sangbum Kim, Enkhbayar Erdenee, and Buru Chang. 2022. Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterances. arXiv preprint arXiv:2204.10825 (2022). - · arXiv
  26. [28][28] Xuanli He, Lingjuan Lyu, Lichao Sun, and Qiongkai Xu. 2021. Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . 2006-2012. -
  27. [29][29] Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring Coding Challenge Competence With APPS. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track . -
  28. [30][30] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 (2019). - · arXiv
  29. [31][31] Zhiqiang Hu, Roy Ka-Wei Lee, Charu C Aggarwal, and Aston Zhang. 2020. Text Style Transfer: A Review and Experimental Evaluation. arXiv preprint arXiv:2010.12742 (2020). - · arXiv
  30. [32][32] Gabbrielle Johnson. 2022. Are Algorithms Value-Free? Feminist Theoretical Virtues in Machine Learning. Journal of Moral Philosophy, special issue on "Justice, Power, and the Ethics of Algorithmic Decision-Making" (2022). -
  31. [34][34] Oscar Karnalim, Setia Budi, Hapnes Toba, and Mike Joy. 2019. Source Code Plagiarism Detection in Academia with Information Retrieval: Dataset and the Observatjion. Informatics in Education 18, 2 (2019), 321-344. -
  32. [35][35] Oscar Karnalim, Judy Sheard, Ilir Dema, Amey Karkare, Juho Leinonen, Michael Liut, and Renée McCauley. 2020. Choosing code segments to exclude from code similarity detection. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education . 1-19. -
  33. [36][36] Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit Iyyer. 2020. Thieves on Sesame Street! Model Extraction of BERT-based APIs. In International Conference on Learning Representations . -
  34. [37][37] Thomas Lancaster and Fintan Culwin. 2004. A comparison of source code plagiarism detection engines. Computer Science Education 14, 2 (2004), 101-112. -
  35. [39][39] Mina Lee, Percy Liang, and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. arXiv preprint arXiv:2201.06796 (2022). - · arXiv
  36. [40][40] Yoonjoo Lee, Tae Soo Kim, Minsuk Chang, and Juho Kim. 2022. Interactive Children's Story Rewriting Through Parent-Children Interaction. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022) . 62-71. -
  37. [41][41] Wei Li, Xinyan Xiao, Jiachen Liu, Hua Wu, Haifeng Wang, and Junping Du. 2020. Leveraging Graph to Improve Abstractive Multi-Document Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . 6232-6243. -
  38. [42][42] Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021). - · arXiv
  39. [43][43] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021). - · arXiv
  40. [44][44] Yang Liu and Mirella Lapata. 2019. Text Summarization with Pretrained Encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . 3730-3740. -
  41. [45][45] Divya Luke, PS Divya, Sony L Johnson, S Sreeprabha, and Elizabeth B Varghese. 2014. Software plagiarism detection techniques: A comparative study. In 2018 5th International Symposium on Emerging Trends and Technologies in Libraries and Information Services (ETTLIS) . Citeseer. -
  42. [46][46] Rohan Mukherjee, Yeming Wen, Dipak Chaudhari, Thomas Reps, Swarat Chaudhuri, and Chris Jermaine. 2021. Neural Program Generation Modulo Static Analysis. In Thirty-Fifth Conference on Neural Information Processing Systems . -
  43. [47][47] Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Ranking Sentences for Extractive Summarization with Reinforcement Learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) . 1747-1759. -
  44. [48][48] Jonas Oppenlaender. 2022. Prompt Engineering for Text-Based Generative Art. arXiv preprint arXiv:2204.13988 (2022). - · arXiv
  45. [49][49] Dieter Pawelczak. 2018. Benefits and drawbacks of source code plagiarism detection in engineering education. In 2018 IEEE Global Engineering Education Conference (EDUCON) . IEEE, 1048-1056. -
  46. [50][50] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). - · arXiv
  47. [51][51] Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems . 1-7. -
  48. [52][52] Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2022. Multitask Prompted Training Enables Zero-Shot Task Generalization. In The Tenth International Conference on Learning Representations . -
  49. [53][53] Dana Sheahen and David Joyner. 2016. TAPS: A MOSS extension for detecting software plagiarism at scale. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale . 285-288. -
  50. [55][55] Avi Shporer, Sunny Tran, Nikhil Singh, Brandon Kates, Jayson Lynch, and Iddo Drori. 2021. Learning Methods for Solving Astronomy Course Problems. preprint (2021). https://www.cs.columbia.edu/~idrori/Learning\_Methods\_for\_Solving\_ Astronomy\_Course\_Problems.pdf -
  51. [56][56] Arjun Singh, Sergey Karayev, Kevin Gutowski, and Pieter Abbeel. 2017. Gradescope: a fast, flexible, and fair system for scalable assessment of handwritten work. In Proceedings of the fourth (2017) acm conference on learning@ scale . 81-88. -
  52. [57][57] Sunny Tran, Pranav Krishna, Ishan Pakuwal, Prabhakar Kafle, Nikhil Singh, Jayson Lynch, and Iddo Drori. 2021. Solving machine learning problems. In Asian Conference on Machine Learning . PMLR, 470-485. -
  53. [58][58] Ted Underwood. 2022. Mapping the Latent Spaces of Culture. To appear in Startwords (2022). https://doi.org/10.17613/faaa-1r21 - · DOI
  54. [59][59] Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts . 1-7. -
  55. [61][61] Justin D Weisz, Michael Muller, Steven I Ross, Fernando Martinez, Stephanie Houde, Mayank Agarwal, Kartik Talamadupula, and John T Richards. 2022. Better together? an evaluation of ai-supported code translation. In 27th International Conference on Intelligent User Interfaces . 369-391. -
  56. [62][62] Zhuohao Wu, Danwen Ji, Kaiwen Yu, Xianxu Zeng, Dingming Wu, and Mohammad Shidujaman. 2021. AI Creativity and the Human-AI Co-creation Model. In International Conference on Human-Computer Interaction . Springer, 171-190. -
  57. [63][63] Frank F Xu, Uri Alon, Graham Neubig, and Vincent J Hellendoorn. 2022. A systematic evaluation of large language models of code. arXiv preprint arXiv:2202.13169 (2022). - · arXiv
  58. [64][64] Muhammad Yasid, Gomgom T.P. Siregar, and Muhammad Ansori Lubis. 2020. The Policy of Credit Payment Relaxation in Overcoming the Impact of Covid-19 Spread to the Economic Society in Indonesia. Journal of Advanced Research in Dynamical and Control Systems (2020). -
  59. [65][65] Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces . 841-852. -
  60. [66][66] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. OPT: Open Pre-trained Transformer Language Models. arXiv preprint arXiv:2205.01068 (2022). · arXiv

Cited by

Sorted by citations

Authors
Biderman, S., & Raff, E.
Affiliations
Booz Allen Hamilton (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Pages
2933-2943
ISSN
2155-0751 (ISSN-L)
DOI
10.48550/arxiv.2201.07406
arXiv
2201.07406
OpenAlex
W4306317681
Cited by
37
References
0
Citations by year
2026: 2 · 2025: 3 · 2024: 17 · 2023: 14 · 2022: 1
FWCI
8.70
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM International Conference on Information and Knowledge Management (CIKM) (publishedVersion, cc-by-sa)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
bryophytes, GPT-J, large language models, code plagiarism detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review