More human than human? Differences in lexis and collocation within academic essays produced by ChatGPT-3.5 and human L2 writers
Open accessAbstract
ChatGPT and similar generative AI models are increasingly being integrated into educational practice, including second language (L2) writing. However, AI-generated writing may significantly differ from that of developing L2 writers, raising potential issues over academic integrity. With AI detectors still underused in many contexts due to accuracy concerns, it is essential to determine if and how AI-generated texts are similar or different to those of L2 writers. This study compares lexical and collocation choices between essays on the same topic generated by ChatGPT (version 3.5) and human L2 writers using corpus analytic techniques. R statistical software was used for analysis and comparison. Results suggest that when tasked with producing academic essays under the same topic, ChatGPT-3.5 excels in generating texts with formal and complex vocabulary suited for academic and technical themes, while human L2 writers tend to focus on personal and social issues, using more varied and context-rich vocabulary. Accordingly, educators seeking to detect AI use should recognise the distinct linguistic and cultural experiences L2 writers bring to their writing, while students using AI need to understand the nature of the machine-like clarity and directness of expression as produced by ChatGPT as compared with their own typical production.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
At the level of collocation and lexical patterning, human EFL writing and AI output differ in ways that reflect distinct processes generating each text type.
“at the level of collocation and lexical patterning, human EFL writing and AI output differ in ways that reflect the distinct processes generating each text type”
From ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY (2026) · cites this paper as M. Zhang & Crosthwaite, 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1× · see citing works
“Generative AI (GenAI) in this study refers to software tools utilising Large Language Models (LLMs) trained on extensive text data that are able to generate human-like text, answer questions, and perform various language-related tasks with high accuracy (Kasneci et al. 2023).”
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · see citing works
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079 · see citing works
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · see citing works
Students' voices on generative ai: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · arXiv · doi:10.48550/arxiv.2305.00290 · Cited by 2,504 · cited 2× · see citing works
“For instance, GenAI can collaborate with teachers to accomplish tasks that are challenging for teachers to complete alone, such as grading a large volume of student assignments and providing the necessary support and sca ff oldings for learners, including real-time feedback (Chan and Hu 2023).”
Students’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 2× · see citing works
“For instance, GenAI can collaborate with teachers to accomplish tasks that are challenging for teachers to complete alone, such as grading a large volume of student assignments and providing the necessary support and sca ff oldings for learners, including real-time feedback (Chan and Hu 2023).”
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003 · cited 2× · see citing works
“…of generative AI tools and their increasing integration into educational contexts, it is imperative to adopt a critical perspective towards these technologies and educate leaners on their appropriate and ethical use, as under-scored by recent research (e.g., Guo et al. 2023; Tlili et al. 2023).”
Learning Vocabulary in Another Language
Nation, I. S. P., & Hunston, S. (2013) · doi:10.1017/cbo9781139858656 · Cited by 1,622 · see citing works
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123618 · Cited by 1,452 · see citing works
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384 · see citing works
Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence
Cooper, G. (2023) · Journal of Science Education and Technology · doi:10.1007/s10956-023-10039-y · Cited by 1,263 · cited 1× · see citing works
“Cooper (2023) highlighted that educators can gain a deeper understanding of the learning process by incorporating GenAI across multiple domains. e.g., providing a mentor for each learner, helping learners and teachers acquire new 21st-century skills, enhancing data interaction, enabling the use of…”
Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation
Yan, D. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11742-4 · Cited by 713 · cited 1× · see citing works
“…include the impact of ChatGPT ’ s automatic text generation on undergraduate students ’ perceptions and behaviours in learning L2 writing skills (Yan 2023), or studies focusing on the use of ChatGPT as a tool for in fl uencing and guiding responses in higher-education students ’ L2 writing (Zou…”
Crawford, J., Cowling, M., & Allen, K.-A. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.3.02 · Cited by 547 · cited 1× · see citing works
“Studies suggest that traditional university assignments are in danger as ChatGPT can now generate high-scoring essays and provide correct answers to problem questions (Al-Zahrani 2023; Crawford et al. 2023).”
AI bot ChatGPT writes smart essays — should professors worry?
Stokel-Walker, C. (2022) · Nature · doi:10.1038/d41586-022-04397-7 · Cited by 543 · see citing works
Comparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509 · cited 1× · see citing works
“LLMs allow GenAI to generate linguistic content similar to humans by learning from extensive training data (Steiss et al. 2024).”
Ädel, A., & Erman, B. (2011) · English for Specific Purposes · doi:10.1016/j.esp.2011.08.004 · Cited by 413 · see citing works
The role of individual differences in L2 writing
Kormos, J. (2012) · Journal of Second Language Writing · doi:10.1016/j.jslw.2012.09.003 · Cited by 396 · see citing works
Barrett, A., & Pack, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00427-0 · Cited by 391 · see citing works
A large-scale comparison of human-written versus ChatGPT-generated essays
Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-45644-9 · Cited by 338 · cited 2× · see citing works
“Advances in LLM technology are also closing the gap, given ChatGPT-4 shows signi fi cantly higher lexical diversity when compared with ChatGPT-3 (Herbold et al. 2023).”
Wang, X., Li, L., Tan, S. C., Yang, L., & Lei, J. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107798 · Cited by 336 · see citing works
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
Guo, B., Zhang, X., Wang, Z., Jiang, M., Nie, J., Ding, Y., Yue, J., & Wu, Y. (2023) · arXiv · doi:10.48550/arxiv.2301.07597 · Cited by 318 · see citing works
Beyond single words: the most frequent collocations in spoken English
Shin, D., & Nation, P. (2007) · ELT Journal · doi:10.1093/elt/ccm091 · Cited by 265 · see citing works
Yang, H., & Kyun, S. (2022) · Australasian Journal of Educational Technology · doi:10.14742/ajet.7492 · Cited by 208 · see citing works
The death of the short-form physics essay in the coming AI revolution
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. P. (2023) · Physics Education · doi:10.1088/1361-6552/acc5cf · Cited by 199 · see citing works
A Review of Best Practice Recommendations for Text Analysis in R (and a User-Friendly App)
Banks, G. C., Woznyj, H. M., Wesslen, R. S., & Ross, R. L. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-017-9528-3 · Cited by 194 · see citing works
Zhao, X. (2023) · RELC Journal · doi:10.1177/00336882221094089 · Cited by 192 · see citing works
Better by You, better than Me? ChatGPT-3 as writing assistance in students’ essays.
Bašić, Ž., Banovac, A., Kružić, I., & Jerković, I. (2023) · doi:10.35542/osf.io/n5m7s · Cited by 189 · see citing works
The impact of ChatGPT on L2 writing and expected responses: Voice from doctoral students
Zou, M., & Huang, L. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12397-x · Cited by 139 · see citing works
Generative AI and the end of corpus-assisted data-driven learning? Not so fast!
Crosthwaite, P. R., & Baisa, V. (2023) · Applied Corpus Linguistics · doi:10.1016/j.acorp.2023.100066 · Cited by 135 · see citing works
Self-citationAl-Zahrani, A. M. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2271445 · Cited by 120 · see citing works
Allen, T. J., & Mizumoto, A. (2024) · RELC Journal · doi:10.1177/00336882241262533 · Cited by 120 · see citing works
Frequent Collocations and Authorial Style
Hoover, D. L. (2003) · Literary and Linguistic Computing · doi:10.1093/llc/18.3.261 · Cited by 90 · see citing works
Does ChatGPT Argue Like Students? Bundles in Argumentative Essays
Jiang, F. (., & Hyland, K. (2025) · Applied Linguistics · doi:10.1093/applin/amae052 · Cited by 76 · cited 3× · see citing works
“Yet , while studies exploring the nature of GenAI academic vocabulary are now coming to the fore (e.g. Jiang and Hyland 2024), the fi eld still lacks research investigating lexical and collocation di ff erences between texts generated by ChatGPT and human L2 writers.”
InfluentialLi, H., Wang, Y., Luo, S., & Huang, C. (2024) · Journal of Asian Public Policy · doi:10.1080/17516234.2024.2363128 · Cited by 71 · see citing works
AlAfnan, M. A., & MohdZuki, S. F. (2023) · Journal of Artificial Intelligence and Technology · doi:10.37965/jait.2023.0267 · Cited by 52 · see citing works
ChatGPT as a COBUILD lexicographer
Lew, R. (2023) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-02119-6 · Cited by 49 · see citing works
Understanding Leadership in Education: Life Writing and its Possibilities
English, F. W. (2006) · Journal of Educational Administration & History · doi:10.1080/00220620600555121 · Cited by 29 · see citing works
ChatGPT and the future of impact assessment
Khan, M., Chaudhry, M. N., Ahsan, M., & Ahmad, R. (2024) · Environmental Science & Policy · doi:10.1016/j.envsci.2024.103779 · Cited by 28 · see citing works
Wilson, J., Myers, M. C., & Potter, A. A. (2022) · Assessment in Education Principles Policy and Practice · doi:10.1080/0969594x.2022.2025762 · Cited by 26 · see citing works
Xia, W., Mao, S., & Zheng, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.03401 · Cited by 23 · see citing works
Xia, D., Chen, Y., & Pae, H. K. (2022) · IRAL - International Review of Applied Linguistics in Language Teaching · doi:10.1515/iral-2021-0188 · Cited by 22 · see citing works
A comparative analysis of multiword units in the reading and listening input of English textbooks
Hoang, H., & Crosthwaite, P. R. (2024) · System · doi:10.1016/j.system.2024.103224 · Cited by 20 · see citing works
Self-citationThe Impact of ChatGPT on Language Evolution: A Linguistic Analysis
Rafique, H., Nazeer, I., & Rehman, J. (2024) · Journal of Education and Social Studies · doi:10.52223/jess.2024.5106 · Cited by 17 · see citing works
Beyond lexical frequencies: using R for text analysis in the digital humanities
Arnold, T., Ballier, N., Lissón, P., & Tilton, L. (2019) · Language Resources and Evaluation · doi:10.1007/s10579-019-09456-6 · Cited by 10 · see citing works
The Influence of Collocation Segmentation and Top 10 Items to Keyword Assignment Performance
Daudaravičius, V. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12116-6_55 · Cited by 10 · see citing works
Understanding the Concepts of Tools and Techniques for Data Analysis Using RStudio
Kumar, L., & Tyagi, V. (2024) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/979-8-3693-3609-0.ch008 · Cited by 7 · see citing works
Zih, H., Biadi, M. E., & Chatri, Z. (2020) · IEEE Congress on Information Science and Technology (CiSt) · doi:10.1109/cist49399.2021.9357285 · Cited by 7 · see citing works
Verzani, J. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315373089-1 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Demir, Cuneyt. 2017. Lexical collocations in English: A comparative study of native and non-native scholars of English. The Journal of Language and Linguistic Studies 13(1). 75–87.
Cited by
Sorted by citations
Generative AI and Second Language Writing
Li, S. (2025) · Digital studies in language and literature · doi:10.1515/dsll-2025-0007 · Cited by 74 · see citing works
Mo, Z., & Crosthwaite, P. R. (2025) · Journal of English for Academic Purposes · doi:10.1016/j.jeap.2025.101499 · Cited by 71 · see citing works
Self-citationEnhancing academic writing in English language education through generative AI integration
Zamorano, C. (2025) · Research Studies in English Language Teaching and Learning · doi:10.62583/rseltl.v3i3.87 · Cited by 9 · see citing works
Engagement strategies in human-written and AI-generated academic essays: A corpus-based study
Alghazo, S. M., Rabab'ah, G. A., El‐Dakhs, D. A. S., & Mustafa, A. (2025) · Ampersand · doi:10.1016/j.amper.2025.100237 · Cited by 8 · see citing works
A Qualitative Approach to EFL Postgraduates’ GenAI-Assisted Research Writing Within Social Sciences
Fuentes, A. B. C. (2025) · Education Sciences · doi:10.3390/educsci15111521 · Cited by 8 · see citing works
Hampson, T. (2026) · Digital studies in language and literature · doi:10.1515/dsll-2026-0012 · Cited by 6 · see citing works
Reframing L2 Academic Writer Identity with GenAI: The Emergence of a Technology-Mediated Self
Dikilitaş, K., & Tavares, V. (2026) · Digital studies in language and literature · doi:10.1515/dsll-2026-0008 · Cited by 3 · see citing works
GenAI and BDDL Tools for Academic L2 English Postgraduate Writing in Tourism: A Local Case Study
Fuentes, A. B. C. (2025) · US-China Education Review A · doi:10.17265/2161-623x/2025.08.002 · Cited by 3 · see citing works
AI-generated, L2 learner, and native German writing: a comparative analysis of linguistic complexity
Sun, S., & Li, Y. (2025) · Glottotheory · doi:10.1515/glot-2025-2011 · Cited by 2 · see citing works
From ‘Nativelike’ to Practical: Comparing Human and AI Reformulation as Written Feedback
Nakai, S., & Shintani, N. (2025) · RELC Journal · doi:10.1177/00336882251397735 · Cited by 2 · see citing works
Zhang, Y. (2025) · International Journal of Applied Linguistics · doi:10.1111/ijal.70071 · Cited by 2 · see citing works
Psychometrics and Linguistics in ChatGPT-Generated Reading Tests Compared with CET-4
Li, B., Qin, Q., Li, W., & Mao, C. (2026) · Journal of Psychoeducational Assessment · doi:10.1177/07342829261444828 · Cited by 2 · see citing works
Almulla, N. (2025) · Open Journal of Modern Linguistics · doi:10.4236/ojml.2025.155044 · Cited by 2 · see citing works
Fuentes, A. B. C. (2025) · Open Journal of Modern Linguistics · doi:10.4236/ojml.2025.156054 · Cited by 1 · see citing works
Turós, M., Soltész-Várhelyi, K., & Szűts, Z. (2026) · Education Sciences · doi:10.3390/educsci16060920 · Cited by 1 · see citing works
Qiu, W., & Pan, F. (2026) · Scientometrics · doi:10.1007/s11192-026-05701-2 · Cited by 1 · see citing works
ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY
Patty, J., & Matatula, M. Y. (2026) · J-Shelves of Indragiri (JSI) · doi:10.61672/jsi.v8i1.3557
“The overlap, however, is not total: at the level of collocation and lexical patterning, human EFL writing and AI output differ in ways that reflect the distinct processes generating each text type (M. Zhang & Crosthwaite, 2025).”
Conversational AI Teaching Assistant for Second Language Writing
Çelık, F., & Duran, V. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3316-8.ch006
Korpusi za učenje srpskog jezika kao stranog u eri veštačke inteligencije
Perišić, O. (2025) · Филолошки факултет Универзитета у Београду eBooks · doi:10.18485/judig.2025.1.ch16
Demir, N. Y. (2026) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqag084
Li, M. (2026) · Journal of English Language Teaching and Applied Linguistics · doi:10.32996/jeltal.2026.8.9.1
AI Untuk Karya Ilmiah Dosen: Menjaga Mutu dan Keaslian
Halawa, I. K. (2025) · Jurnal Sosiologi Agama dan Teologi Indonesia · doi:10.24246/sami.vol3i2pp429-444
Academic vocabulary use in research publications in the AI era: human-authored vs AI-mediated texts
Laosrirattanachai, P., & Laosrirattanachai, P. (2026) · The JALT CALL Journal · doi:10.29140/jaltcall.2026.103271
Öksüz, H. İ., Afyoncu, T., Gök, S., Yıldırım, T., & Yıldırım, K. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2734314
Engagement Functions of AI-Generated Text from the Perspective of Appraisal Theory
Sun, H. (2026) · International Journal of English Literature and Social Sciences · doi:10.22161/ijels.115.28
Do LLMs “Normalize” Abstracts? Evidence From Paired Sequence and Position Analyses of Santos Moves
Qiao, F., Wu, Q., & Zhou, Y. (2026) · SAGE Open · doi:10.1177/21582440261499856
- Authors
- Zhang, M., & Crosthwaite, P.
- Affiliations
- The University of Queensland (AU)
- Corresponding author
- Crosthwaite, Peter
- ORCID
- Crosthwaite, Peter
- Venue
- International Review of Applied Linguistics in Language Teaching
- Publisher
- Walter de Gruyter GmbH
- Published
- 2025-01-23
- Volume
- 64
- Issue
- 2
- Pages
- 733-760
- ISSN
- 0019-042X (ISSN-L)
- 1613-4141
- OpenAlex
- W4406702912
- Semantic Scholar
- 93f66ee18565842b6c79b4abcba1d16508bff4a4
- Cited by
- 26
- References
- 47
- Citations by year
- 2026: 12 · 2025: 14
- FWCI
- 28.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IRAL - International Review of Applied Linguistics in Language Teaching (publishedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Discourse Analysis in Language Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- ChatGPT-3.5, AI-generated text, second language writing, word choice, collocations, academic essays, AI detection, academic integrity, vocabulary complexity, generative AI in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
