CoChat
Sign Up
Cited by 38Academic integrity and plagiarismArtificial Intelligence in EducationAuthorship Attribution and Profiling

Who Wrote This?: Detecting Artificial Intelligence–Generated Text from Human-Written Text

Open access
Kumar, Rahul, Mindzak, Michael·2024·Canadian Perspectives on Academic Integrity
DOI 10.55016/ojs/cpai.v7i1.77675
Open this paper in CoChat — free accountOpen PDF

Abstract

This article explores the impact of artificial intelligence (AI) on written compositions in education. The study examines participants’ accuracy in distinguishing between texts generated by humans and those produced by generative AI (GenAI). The study challenges the assumption that the listed author of a paper is the one who wrote it, which has implications for formal educational systems. If GenAI text becomes indistinguishable from human-generated text to a human instructor, marker, or grader, it raises concerns about the authenticity of submitted work. This is particularly relevant in post-secondary education, where academic papers are crucial in assessing students’ learning, application, and reflection. The study had 135 participants who were randomly presented with two passages in one session. The passages were on the topic of “How will technology change education?” and were placed into one of three pools based on the source of origin: written by researchers, generated by AI, and searched and copied from the internet. The study found that participants were able to identify human-generated texts with an accuracy rate of 63%. But with an accuracy of only 24% when the composition was AI-generated. However, the study also had limitations, such as limited sample size and an older predecessor of the current GenAI software. Overall, this study highlights the potential impact of AI on education and the need for further research to evaluate comparisons between AI-generated and human-generated text.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.55016/ojs/cpai.v7i1.77675

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 35 · Cited by 38

Research works printed but not linked: 23

  1. Abd-Elaal, E., Gamage, S. H. P. W., & Mills, J. E. (2019). Artificial intelligence is a tool for cheat- ing academic integrity [paper presentation. In Aaee 2019 annual conference. Queensland, Aus- tralia. https://aaee.net.au/wp-content/uploads/2020/07/ AAEE2019 Annual Conference paper 180.pdf
  2. Abd-Elaal, E., Gamage, S. H. P. W., & Mills, J. E. (2022). Assisting academics to identify computer gener- ated writing. European Journal of Engineering Education, 47(5), 725-745. https://doi.org/10.1080/03043797.2022 .2046709 · DOI · Paper page
  3. Brinkman, B. (2013). An analysis of student privacy rights in the use of plagiarism detection systems. Science and En- gineering Ethics, 19, 1255-1266. https://doi.org/10.1007/ s11948-012-9370-y · DOI
  4. Curtis, G. J., Slade, C., Bretag, T., & McNeill, M. (2021). Developing and evaluating nationwide expert-delivered aca- demic integrity workshops for the higher education sector in australia. Higher Education Research and Development, 41(3), 665-680. https://doi.org/10.1080/07294360.2021 .1872057 · DOI
  5. Dawson, P., & Sutherland-Smith, W. (2018a). Can markers detect contract cheating? results from a pilot study. Assess- ment and Evaluation in Higher Education, 43(2), 286-293. https://doi.org/10.1080/02602938.2017.1336746 · DOI
  6. Dawson, P., & Sutherland-Smith, W. (2018b). Can train- ing improve marker accuracy at detecting contract cheat- ing? a multi-disciplinary pre-post study. Assessment and Evaluation in Higher Education, 44(5), 715-725. https://doi.org/10.1080/02602938.2018.1531109 · DOI
  7. Drake, C. A. (1941). Why students cheat. The Journal of Higher Education, 12(8), 418-420. https://doi.org/10.1080/ 00221546.1941.11773211 · DOI
  8. Eaton, S. E., & Christensen-Hughes, J. (Eds.). (2022). Aca- demic integrity in canada: An enduring and essential chal- lenge. Springer. https://doi.org/10.1007/978-3-030-83255 -1 · DOI
  9. Gray, B. C. (2022). Ethics, edtech, and the rise of con- tract cheating. In J. Eaton S.E. & Hughes (Ed.), Aca- demic integrity in canada: An enduring and essential chal- lenge (p. 189-201). Springer. https://doi.org/10.1007/ 978-3-030-83255-1 9 · DOI
  10. Haman, M., & Školník, M. (2023). Using chatgpt to conduct a literature review. Accountability in Research. https:// doi.org/10.1080/08989621.2023.2185514 · DOI · Paper page
  11. Harper, R., Bretag, T., & Rundle, K. (2021). Detect- ing contract cheating: Examining the role of assessment type. Higher Education Research and Development, Who wrote this? Detecting artificial intelligence-generated text from human-written text -8 40(2), 263-278. https://doi.org/10.1080/07294360.2020 .1724899 · DOI
  12. Jordan, A. E. (2001). College student cheating: The role of motivation, perceived norms, attitudes, and knowledge of institutional policy. Ethics and Behavior, 11(3), 233-247. https://www.doi.org/10.1207/S15327019EB1103 3 · DOI
  13. Kerr, K. (2020). Ethical considerations when using artificial intelligence-based assistive technologies in education. In B. Brown, V. Roberts, M. Jacobsen, & C. Hurrell (Eds.), Ethical use of technology in digital learning environments: Graduate student perspectives (p. 9-14). University of Calgary. https://doi.org/10.7939/r3-yanz-fe63 · DOI
  14. Köbis, N., & Mossink, L. D. (2021). Artificial intelligence versus maya angelou: Experimental evidence that people cannot differentiate ai-generated from human-written po- etry. Computers in Human Behavior, 114, Article 106553. https://doi.org/10.1016/j.chb.2020.106553 · DOI · Paper page
  15. Lancaster, T. (2023). Artificial intelligence, text generation tools and chatgpt-does digital watermarking offer a solu- tion? International Journal for Educational Integrity, 19, Article 10. https://doi.org/10.1007/s40979-023-00131-6 · DOI · Paper page
  16. Luitse, D., & Denkena, W. (2021). The great transformer: Examining the role of large language models in the political economy of ai. Big Data & Society, 8(2). https://doi.org/ 10.1177/20539517211047734 · DOI
  17. McKnight, L. (2021). Electric sheep? humans, robots, ar- tificial intelligence, and the future of writing. Studies in Culture and Education, 28(4), 442-455. https://doi.org/ 10.1080/1358684X.2021.1941768 · DOI · Paper page
  18. Perkins, M., Roe, J., Postma, D., McGaughran, J., Hickerson, D., & Cook, J. (2023). Game of tones: Faculty detection of gpt-4 generated content in university assessments. arXiv. https://www.doi.org/10.48550/arXiv.2305.18081 · DOI · Paper page
  19. Selwyn, N. (2019). Should robots replace teachers? ai and the future of education. Wiley.
  20. Southgate, E. (2021). Artificial intelligence and machine learning: A practical and ethical guide for teachers. In C. Wyatt-Smith, B. Lingard, & E. Heck (Eds.), Digital disruption in teaching and testing: Assessments, big data, and the transformation of schooling (2nd ed., p. 60-74). Routledge. https://www.doi.org/10.4324/9781003045793 -3 · DOI
  21. Vrbanec, T., & Meštrović, A. (2021). Taxonomy of academic plagiarism methods. Zbornik Veleučilišta u Rijeci, 9(1), 283-300. http://doi.org/10.31784/zvr.9.1.17 · DOI · Paper page
  22. Williamson, B., & Enyon, R. (2020). Historical trends, miss- ing links, and future directions in ai in education. Learning, Media and Technology, 45(3), 223-235. https://doi.org/ 10.1080/17439884.2020.1798995 Who wrote this? Detecting artificial intelligence-generated text from human-written text -9 · DOI · Paper page
  23. Yang, X., Li, Y., Zhang, X., Chen, H., & Cheng, W. (2023). Exploring the limits of chatgpt for query or aspect-based text summarization. https://doi.org/10.48550/arXiv.2302 .08081 (arXiv.) · DOI

Authors
Kumar, R., & Mindzak, M.
Affiliations
Brock University (CA)
ORCID
Venue
Canadian Perspectives on Academic Integrity
Publisher
University of Calgary
Published
2024-01-10
Volume
7
Issue
1
ISSN
2561-6218 (ISSN-L)
DOI
10.55016/ojs/cpai.v7i1.77675
OpenAlex
W4390690726
Cited by
38
References
35
Citations by year
2026: 15 · 2025: 16 · 2024: 7
FWCI
15.82
Type
journal article
Language
en
Open access
Licence
cc-by-nc-nd
Locations
Canadian Perspectives on Academic Integrity (publishedVersion, cc-by-nc-nd)
Funders
Deakin University
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
Keywords
generative AI, AI-generated text, academic integrity, authorship attribution, student writing, academic assessment, postsecondary education, textual authenticity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review