Detecting ChatGPT-generated essays in a large-scale writing assessment: Is there a bias against non-native English speakers?
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Detection models trained on balanced datasets including both native and non-native writing samples have demonstrated substantially reduced bias without major loss in overall accuracy.
“Detection models trained on balanced datasets that include both native and non-native writing samples have demonstrated substantially reduced bias without major loss in overall accuracy”
From ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY (2026) · cites this paper as Jiang et al., 2024
This bias is not incidental but structurally embedded in the design and training of these systems.
“This bias is not incidental but structurally embedded in the design and training of these systems (Caner, 2025; Ibrahim, 2023; Jiang et al., 2024; Liang et al., 2023).”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Jiang et al., 2024
The launch of ChatGPT is widely regarded as a milestone in the mainstream adoption of GenAI.
“Although ChatGPT was not the first publicly available AI chatbot, we used its launch as the cut-off point because it is widely regarded as a milestone in the mainstream adoption of GenAI 22,23,24,25 .”
From Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports (2025) · cites this paper as [24]
A more recent study found that non-native texts had the highest perplexity, while machine-generated texts consistently had much lower perplexity, and using this feature alone achieved 99.9% accuracy in detecting MGTs.
“Contrary to the earlier claims, this analysis showed that non-native texts had the highest perplexity, while MGTs consistently had much lower perplexity. Using this feature alone, the authors reported 99.9% accuracy in detecting MGTs.”
From AI for scientific integrity: detecting ethical breaches, errors, and misconduct in manuscripts (2025) · cites this paper as Jiang et al., 2024
Automated assessment systems are biased, further burdening non-native learners.
“Jiang et al. (2024) affirm that automated assessment systems are biased, which further burdens non-native learners.”
From Mapping the impact of generative AI in higher education: A scoping review of psychological and equity dimensions. (2026) · cites this paper as Jiang et al. (2024)
Cites
Effect size estimates: Current use, calculations, and interpretation.
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012) · Journal of Experimental Psychology: General · doi:10.1037/a0024338 · Cited by 5,199 · see citing works
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · see citing works
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079 · see citing works
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · see citing works
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993 · see citing works
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673 · see citing works
Memarian, B., & Doleck, T. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100152 · Cited by 416 · see citing works
All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.565 · Cited by 290 · see citing works
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text
Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021) · arXiv · doi:10.48550/arxiv.2107.00061 · Cited by 290
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2024) · Journal of Academic Ethics · doi:10.1007/s10805-023-09492-6 · Cited by 234 · see citing works
TweepFake: about detecting deepfake tweets
Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251415 · Cited by 185 · see citing works
Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover
Fröhling, L., & Zubiaga, A. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.443 · Cited by 106 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Stone, B. W. (2025) · Teaching of Psychology · doi:10.1177/00986283241305398 · Cited by 54 · see citing works
Opportunities and challenges of using generative AI to personalize educational assessment
Arslan, B., Lehman, B., Tenison, C., Sparks, J. R., López, A. A., Gu, L., & Zapata-Rivera, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1460651 · Cited by 52 · see citing works
Writing with AI: What College Students Learned from Utilizing ChatGPT for a Writing Assignment
Wang, C., Aguilar, S. J., Bankard, J. S., Bui, E., & Nye, B. (2024) · Education Sciences · doi:10.3390/educsci14090976 · Cited by 49 · see citing works
“Despite ongoing research on the automated detection of AI-generated texts [5], it has been difficult for teachers or plagiarism detection tools to distinguish between the texts written by students themselves and the texts generated by ChatGPT [6–8].”
AI for scientific integrity: detecting ethical breaches, errors, and misconduct in manuscripts
Pellegrina, D., & Helmy, M. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1644098 · Cited by 30 · see citing works
AI chatbots: A disguised enemy for academic integrity?
Niloy, A. C., Hafiz, R., Hossain, B. M., Gulmeher, F., Sultana, N., Islam, K. F., Bushra, F., Islam, S., Hoque, S. I., Rahman, M., et al. (2024) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2024.100396 · Cited by 26 · see citing works
Ni, W., & Li, J. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1856854 · Cited by 6 · see citing works
Mak, M. H. C., & Walasek, L. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100507 · Cited by 1 · see citing works
ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY
Patty, J., & Matatula, M. Y. (2026) · J-Shelves of Indragiri (JSI) · doi:10.61672/jsi.v8i1.3557
“Detection models trained on balanced datasets that include both native and non-native writing samples have demonstrated substantially reduced bias without major loss in overall accuracy (Ibrahim et al., 2025; Jiang et al., 2024).”
GİRAY, L., CANER, H. N., & AMOOZEGAR, A. (2026) · Journal of Narrative and Language Studies · doi:10.59045/nalans.2026.109
- Authors
- Jiang, Y., Hao, J., Fauss, M., & Li, C.
- Affiliations
- Educational Testing Service (US)
- Corresponding author
- Jiang, Yang
- Venue
- Computers & Education
- Publisher
- Elsevier BV
- Published
- 2024-08
- Volume
- 217
- Pages
- 105070
- ISSN
- 0360-1315 (ISSN-L)
- OpenAlex
- W4396673785
- Cited by
- 68
- References
- 12
- Citations by year
- 2026: 26 · 2025: 31 · 2024: 11
- FWCI
- 28.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Computers & Education (publishedVersion)
- Funders
- ETS
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- ChatGPT, AI-generated text detection, generative AI in education, non-native English speakers, algorithmic bias, automated essay scoring, linguistic features, academic integrity, AI policy in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
