CoChat
Sign Up
PreprintCited by 116Academic integrity and plagiarismArtificial Intelligence in EducationAuthorship Attribution and Profiling

GenAI Detection Tools, Adversarial Techniques and Implications for Inclusivity in Higher Education

Open access
Perkins, Mike, Roe, Jasper, Vu, Binh H., Postma, Darius, Hickerson, Don, McGaughran, James, Khuat, Huy Q.·2024·arXiv
DOI 10.48550/arxiv.2403.19148arXiv 2403.19148

Preprint — published as 10.1186/s41239-024-00487-w · Paper page

Open this paper in CoChat — free accountOpen PDF

Abstract

This study investigates the efficacy of six major Generative AI (GenAI) text detectors when confronted with machine-generated content that has been modified using techniques designed to evade detection by these tools (n=805). The results demonstrate that the detectors' already low accuracy rates (39.5%) show major reductions in accuracy (17.4%) when faced with manipulated content, with some techniques proving more effective than others in evading detection. The accuracy limitations and the potential for false accusations demonstrate that these tools cannot currently be recommended for determining whether violations of academic integrity have occurred, underscoring the challenges educators face in maintaining inclusive and fair assessment practices. However, they may have a role in supporting student learning and maintaining academic integrity when used in a non-punitive manner. These results underscore the need for a combined approach to addressing the challenges posed by GenAI in academia to promote the responsible and equitable use of these emerging technologies. The study concludes that the current limitations of AI text detectors require a critical approach for any possible implementation in HE and highlight possible alternatives to AI assessment strategies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2403.19148

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 53

  1. Abd-Elaal, E.-S., Gamage, S. H., & Mills, J. E. (2022). Assisting academics to identify computer generated writing. European Journal of Engineering Education. https:// doi. org/ 10. 1080/ 03043 797. 2022. 20467 09 · DOI · Paper page
  2. Alsmadi, I., Ahmad, K., Nazzal, M., Alam, F., Al-Fuqaha, A., Khreishah, A., & Algosaibi, A. (2021). Adversarial Attacks and Defenses for Social Network Text Processing Applications: Techniques, Challenges and Future Research Directions (arXiv: 2110. 13980). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2110. 13980 · DOI
  3. Amano, T., Ramírez-Castañeda, V., Berdejo-Espinola, V., Borokini, I., Chowdhury, S., Golivets, M., González-Trujillo, J. D., Montaño-Centellas, F., Paudel, K., White, R. L., & Veríssimo, D. (2023). The manifold costs of being a non-native English speaker in science. PLOS Biology. https:// doi. org/ 10. 1371/ journ al. pbio. 30021 84 · DOI
  4. Anderson, N., Belavy, D. L., Perle, S. M., Hendricks, S., Hespanhol, L., Verhagen, E., & Memon, A. R. (2023). AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in Sports & Exercise Medicine manuscript generation. BMJ Open Sport & Exercise Medicine, 9(1), e001568. https:// doi. org/ 10. 1136/ bmjsem-2023-001568 · DOI · Paper page
  5. Bedington, A., Halcomb, E. F., McKee, H. A., Sargent, T., & Smith, A. (2024). Writing with generative AI and human-machine teaming: Insights and recommendations from faculty and students. Computers and Composition, 71, 102833. https:// doi. org/ 10. 1016/j. compc om. 2024. 102833 · DOI
  6. Bissessar, C. (2023). To use or not to use ChatGPT and assistive artificial intelligence tools in higher education institutions? The modern-day conundrum-students' and faculty's perspectives. Equity in Education & Society, https:// doi. org/ 10. 1177/ 27526 46123 12150 · DOI · Paper page
  7. Borji, A., & Mohammadian, M. (2023). Battle of the Wordsmiths: Comparing ChatGPT, GPT-4, Claude, and Bard (SSRN Scholarly Paper 4476855). https:// doi. org/ 10. 2139/ ssrn. 44768 55 · DOI
  8. Chaka, C. (2023a). Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: The case of five AI content detection tools. Journal of Applied Learning and Teaching, 6(2), 2. https:// doi. org/ 10. 37074/ jalt. 2023.6. 2. 12 · DOI · Paper page
  9. Chaka, C. (2023b). Generative AI Chatbots-ChatGPT versus YouChat versus Chatsonic: Use Cases of Selected Areas of Applied English Language Studies. International Journal of Learning, Teaching and Educational Research, 22(6), 6. https:// ijlter. org/ index. php/ ijlter/ artic le/ view/ 7524
  10. Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education. https:// doi. org/ 10. 1186/ s41239-023-00408-3 · DOI · Paper page
  11. Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., & Xie, X. (2023). A Survey on Evaluation of Large Language Models (arXiv: 2307. 03109). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2307. 03109 · DOI · Paper page
  12. Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. https:// doi. org/ 10. 1080/ 14703 297. 2023. 21901 · DOI · Paper page
  13. Curtis, G. J., & Clare, J. (2024). Prevalence, incidence, and rates of contract cheating. In S. E. Eaton (Ed.), Second handbook of academic integrity (pp. 681-696). Springer Nature Switzerland. https:// doi. org/ 10. 1007/ 978-3-031-54144-5_ 110 · DOI · Paper page
  14. Daly, T. M., & Ryan, J. C. (2024). University 'Pay-for-grades': The bait and switch search engine optimization strategies of contract cheating websites in the United States. International Journal for Educational Integrity, 20(1), 1. https:// doi. org/ 10. 1007/ s40979-023-00148-x · DOI · Paper page
  15. Elali, F. R., & Rachid, L. N. (2023). AI-generated research paper fabrication and plagiarism in the scientific community. Pat- terns, 4(3), 100706. https:// doi. org/ 10. 1016/j. patter. 2023. 100706 · DOI · Paper page
  16. Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity, 19(1), 17. https:// doi. org/ 10. 1007/ s40979-023-00140-5 · DOI · Paper page
  17. Fleckenstein, J., Meyer, J., Jansen, T., Keller, S. D., Köller, O., & Möller, J. (2024). Do teachers spot AI? Evaluating the detect- ability of AI-generated texts among student essays. Computers and Education: Artificial Intelligence, 6, 100209. https:// doi. org/ 10. 1016/j. caeai. 2024. 100209 · DOI · Paper page
  18. Foltynek, T., Bjelobaba, S., Glendinning, I., Khan, Z. R., Santos, R., Pavletic, P., & Kravjar, J. (2023). ENAI Recommendations on the ethical use of Artificial Intelligence in Education. International Journal for Educational Integrity. https:// doi. org/ 10. 1007/ s40979-023-00133-4 · DOI · Paper page
  19. Fröhling, L., & Zubiaga, A. (2021). Feature-based detection of automated language models: Tackling GPT-2, GPT-3 and Grover. PeerJ Computer Science, 7, e443. https:// doi. org/ 10. 7717/ peerj-cs. 443 · DOI · Paper page
  20. Fu, Q.-K., Zou, D., Xie, H., & Cheng, G. (2022). A review of AWE feedback: Types, learning outcomes, and implications. Computer Assisted Language Learning. https:// doi. org/ 10. 1080/ 09588 221. 2022. 20337 · DOI
  21. Furze, L., Perkins, M., Roe, J., & MacVaugh, J. (2024). The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment (arXiv: 2403. 14692). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2403. 14692 · DOI
  22. Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers (p. 2022.12.23.521610). bioRxiv. https:// doi. org/ 10. 1101/ 2022. 12. 23. 521610 · DOI · Paper page
  23. Hilliard, L. P., & Stewart, M. K. (2019). Time well spent: Creating a community of inquiry in blended first-year writing courses. The Internet and Higher Education, 41, 11-24. https:// doi. org/ 10. 1016/j. iheduc. 2018. 11. 002 · DOI
  24. Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2020). Automatic Detection of Generated Text is Easiest when Humans are Fooled (arXiv: 1911. 00650). arXiv. https:// doi. org/ 10. 48550/ arXiv. 1911. 00650 · DOI · Paper page
  25. Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023). Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (arXiv: 2303. 13408). arXiv. http:// arxiv. org/ abs/ 2303. 13408
  26. Lancaster, T. (2023). Artificial intelligence, text generation tools and ChatGPT-does digital watermarking offer a solution? International Journal for Educational Integrity, 19(1), 1. https:// doi. org/ 10. 1007/ s40979-023-00131-6 · DOI · Paper page
  27. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers (arXiv: 2304. 02819). arXiv. http:// arxiv. org/ abs/ 2304. 02819
  28. Lin, Y.-T., & Chen, Y.-N. (2023). LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models (arXiv: 2305. 13711). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2305. 13711 · DOI
  29. Lozić, E., & Štular, B. (2023). Fluent but not factual: a comparative analysis of ChatGPT and Other AI Chatbots' proficiency and originality in scientific writing for humanities. Future Internet, 15(10), 10. https:// doi. org/ 10. 3390/ fi151 00336 · DOI
  30. Lutz, C. (2019). Digital inequalities in the age of artificial intelligence and big data. Human Behavior and Emerging Tech- nologies, 1(2), 141-148. https:// doi. org/ 10. 1002/ hbe2. 140 · DOI · Paper page
  31. Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C. (2023). DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (arXiv: 2301. 11305). arXiv. http:// arxiv. org/ abs/ 2301. 11305
  32. Morris, C., Milton, E., & Goldstone, R. (2019). Case study: Suggesting choice: Inclusive assessment processes. Higher Educa- tion Pedagogies, 4(1), 435-447. https:// doi. org/ 10. 1080/ 23752 696. 2019. 16694 · DOI
  33. Orenstrakh, M. S., Karnalim, O., Suarez, C. A., & Liut, M. (2023). Detecting LLM-Generated Text in Computing Education: A Comparative Study for ChatGPT Cases (arXiv: 2307. 07411). arXiv. http:// arxiv. org/ abs/ 2307. 07411
  34. Originality.AI. (2023). AI Content Detector Accuracy Review + Open Source Dataset and Research Tool -Originality.AI. https:// origi nality. ai Paraphrasing Tool-QuillBot AI. (n.d.). Retrieved 3 March 2024, from https:// quill bot. com
  35. Perkins, M. (2023). Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning Practice. https:// doi. org/ 10. 53761/1. 20. 02. 07 · DOI · Paper page
  36. Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024a). The Artificial Intelligence Assessment Scale (AIAS): A Framework for Ethical Integration of Generative AI in Educational Assessment. Journal of University Teaching and Learning Practice, 21(06), 06. https:// doi. org/ 10. 53761/ q3azd e36 · DOI · Paper page
  37. Perkins, M., & Roe, J. (2024a). Academic publisher guidelines on AI usage: A ChatGPT supported thematic analysis [version 2; peer review: 3 approved, 1 approved with reservations]. In F1000Research (Vol. 12, Issue 1398). https:// doi. org/ 10. 12688/ f1000 resea rch. 142411. · DOI
  38. Perkins, M., & Roe, J. (2024b). The use of Generative AI in qualitative analysis: Inductive thematic analysis with ChatGPT. Journal of Applied Learning and Teaching, 7(1), 1. https:// doi. org/ 10. 37074/ jalt. 2024.7. 1. 22 · DOI · Paper page
  39. Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2023). Detection of GPT-4 generated text in higher educa- tion: combining academic judgement and software to identify generative AI tool misuse. Journal of Academic Ethics. https:// doi. org/ 10. 1007/ s10805-023-09492-6 · DOI · Paper page
  40. Qiu, S., Liu, Q., Zhou, S., & Wu, C. (2019). Review of artificial intelligence adversarial attack and defense technologies. Applied Sciences, 9(5), 5. https:// doi. org/ 10. 3390/ app90 50909 · DOI
  41. Radia, P., & Stapleton, P. (2009). Unconventional sources as a new convention: The shifting paradigm of undergraduate writing. The Internet and Higher Education, 12(3), 156-164. https:// doi. org/ 10. 1016/j. iheduc. 2009. 05. 002 · DOI
  42. Roe, J., & Perkins, M. (2022). What are Automated Paraphrasing Tools and how do we address them? A review of a grow- ing threat to academic integrity. International Journal for Educational Integrity, 18(1), 1. https:// doi. org/ 10. 1007/ s40979-022-00109-w · DOI · Paper page
  43. Roe, J., & Perkins, M. (2023). Welcome to the University of life, can I take your order? Investigating Life Experience Degree Offerings in Diploma mills. International Journal for Educational Integrity. https:// doi. org/ 10. 1007/ s40979-023-00138-z · DOI · Paper page
  44. Roe, J., Perkins, M., & Tregubova, Y. (2024). The EAP-AIAS: Adapting the AI Assessment Scale for English for Academic Purposes (arXiv: 2408. 01075). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2408. 01075 · DOI
  45. Rogerson, A. M., & McCarthy, G. (2017). Using Internet based paraphrasing tools: Original work, patchwriting or facilitated plagiarism? International Journal for Educational Integrity, 13(1), 2. https:// doi. org/ 10. 1007/ s40979-016-0013-y · DOI
  46. Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023). Can AI-Generated Text be Reliably Detected? (arXiv: 2303. 11156). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2303. 11156 · DOI · Paper page
  47. Sobaih, A. E. (2024). Ethical concerns for using artificial intelligence chatbots in research and publication: Evidences from Saudi Arabia. Journal of Applied Learning and Teaching, 7(1), 1. https:// doi. org/ 10. 37074/ jalt. 2024.7. 1. 21 · DOI
  48. Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., McCain, M., Newhouse, A., Blazakis, J., McGuffie, K., & Wang, J. (2019). Release Strategies and the Social Impacts of Language Models (arXiv: 1908. 09203). arXiv. https:// doi. org/ 10. 48550/ arXiv. 1908. 09203 · DOI · Paper page
  49. Walters, W. H. (2023). The effectiveness of software designed to detect AI-generated writing: A comparison of 16 AI text detectors. Open Information Science. https:// doi. org/ 10. 1515/ opis-2022-0158 · DOI · Paper page
  50. Wang, W., Wang, R., Wang, L., Wang, Z., & Ye, A. (2023). Towards a robust deep neural network against adversarial texts: A survey. IEEE Transactions on Knowledge and Data Engineering, 35(3), 3159-3179. https:// doi. org/ 10. 1109/ TKDE. 2021. 31176 · DOI
  51. Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), 1. https:// doi. org/ 10. 1007/ s40979-023-00146-z · DOI · Paper page
  52. Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., & Cai, C. J. (2022). PromptChainer: Chaining Large Language Model Prompts through Visual Programming. Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, https:// doi. org/ 10. 1145/ 34911 01. 35197 · DOI · Paper page
  53. Wu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J., & Kim, Y. (2023). Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks (arXiv: 2307. 02477). arXiv. https:// doi. org/ 10. 48550/ arXiv. 2307. 02477 · DOI · Paper page

Cited by

Sorted by citations

Authors
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q.
Affiliations
  • British University Vietnam (VN)
  • James Cook University Singapore (SG)
Corresponding author
Perkins, Mike
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
21
Issue
1
ISSN
2365-9440 (ISSN-L)
DOI
10.48550/arxiv.2403.19148
arXiv
2403.19148
OpenAlex
W4402340204
Cited by
116
References
0
Citations by year
2026: 63 · 2025: 47 · 2024: 6
FWCI
48.30
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • ResearchOnline - JCU (James Cook University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Durham Research Online (Durham University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
Keywords
academic integrity, false accusation, inclusive assessment, higher education, assessment fairness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review