CoChat

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index; 38 of the 55 sources its claims cite were checked for support.

The impact of generative artificial intelligence tools on assessment equity for non-native English-speaking medical students: a systematic review

Open access
Ndacyayisenga, Jean Claude Yvan, Kidega, Charles, Aciro Can, Lucy · 2026 · BMC Medical Education

Abstract

Background: Generative artificial intelligence (GenAI) has been integrated into medical education assessment, creating concerns about fairness toward linguistically diverse students. This systematic review summarized evidence on the effect of GenAI on assessment validity and systematic bias for non-native English speakers (NNES). This study focused on medical students (NNES) who undergo training within the framework of the modern theory of validity. Methods: PRISMA 2020, ERIC, PubMed, Scopus, and Web of Science databases were searched (2022–2025). The included studies assessed GenAI applications (e.g., LLMs, AI detectors, and automated scoring) in tests administered to medical, nursing, pharmacy, or dental students in English-medium institutions. Two reviewers screened, extracted the data, and independently evaluated the quality of the MMAT and AXIS instruments. Thematic synthesis was based on Kane’s validity framework. Results: A total of 1213 were recorded, screened, and 27 studies were included in the sample ( n = 14 quantitative, n = 8 mixed methods, n = 5 qualitative). Six experimental research studies concluded that AI detectors falsely labeled NNES writing as AI-created in 50.2%–61.3% of all cases (compared to less than 5% among native writers). Four automated scoring studies showed a downward bias of 0.5–1.2 SD of NNES to be systematic for students. According to the student survey, GenAI adoption was lower ( n = 8, total = 2,847), with 79% expressing considerable apprehension (63% said it was an issue of concern) and 71% perceived policy ambiguity. Qualitative research showed tension between GenAI as a linguistic aid and the danger of false misconduct accusations, highlighting the need for clear guidelines and support for NNES students to effectively navigate these challenges. Conclusions: Modern GenAI assessment instruments may cause systematic construct-irrelevant variance, putting NNES medical students at a disadvantage. In high-stakes decisions, equity requires multilingual validation, bias audits, transparent governance, and human supervision.

Overview

CoChat Summary

TL;DR

This systematic review of 27 studies finds that generative AI assessment tools can introduce systematic construct-irrelevant variance against non-native English-speaking (NNES) medical students, especially through AI-detector false positives and automated scoring bias. It concludes that equity in high-stakes assessment requires multilingual validation, bias audits, transparent governance, and human oversight.

Key findings

  • Across six experimental studies, AI detectors falsely labeled NNES writing as AI-generated at a median of 55.7% (IQR 51.2–59.3%; abstract range 50.2–61.3%), compared with a median of 3.2% (IQR 1.8–4.7%) for native English speakers; the review states NNES students were approximately 17 times more likely to be falsely accused on a single submission.
  • Four automated scoring studies found a systematic downward bias of 0.5–1.2 SD against NNES students, while native English speakers’ scores showed no such bias.
  • Among 2,847 medical students across eight survey studies, 79% of NNES students expressed considerable apprehension about GenAI in high-stakes assessments, 63% cited it as a concern, and 71% perceived policy ambiguity; only 12% of native speakers reported similar concerns.
  • Qualitative synthesis showed tension between GenAI as a linguistic aid and the danger of false misconduct accusations, with bias attributed to dominant linguistic norms embedded in training corpora rather than to isolated tool bugs.

Methods

  • PRISMA 2020 systematic review; searched ERIC, PubMed, Scopus, and Web of Science (2022–2025) for empirical studies of GenAI assessment tools in medical, nursing, pharmacy, or dental students in English-medium institutions.
  • From 1,213 records, 383 duplicates were removed, 830 were screened, and 27 studies were included (14 quantitative, 8 mixed methods, 5 qualitative); two reviewers screened and extracted data, with 94% agreement on a random 20% sample, and quality was assessed using MMAT and AXIS.
  • Thematic synthesis used Kane’s validity framework and the GAI-AEM conceptual model, organized by PICOS and the review’s three research questions.

Limitations

  • Stated: no longitudinal studies, so the review cannot determine whether GenAI use improves unassisted English proficiency or creates long-term dependency; only English-language publications were included; studies were heavily skewed toward English-dominant countries (78% from USA, Canada, Australia, UAE), limiting generalizability to non-Western settings; and the 2022–2025 window is short, with most included studies being cross-sectional surveys.
  • Stated quality appraisal: 7 studies (25.9%) met only 60–79% of MMAT/AXIS criteria and 2 (7.4%) met less than 60%; common weaknesses were single-institution samples (n = 15), unvalidated instruments (n = 12), no adjustment for confounders (n = 10), and small subgroup samples.
  • Evident from the text: the evidence base is small and methodologically fragile, with only six experimental studies and no long-term outcome data linking false positives or scoring bias to progression or attrition; the review itself notes this causal pathway is plausible but not longitudinally tested.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12909-026-09303-7

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Generative AI assessment equity model (GAI-AEM)
Fig. 1 Generative AI assessment equity model (GAI-AEM)CC BY · PubMed Central
Fig. 2 Prisma Flow 2020
Fig. 2 Prisma Flow 2020CC BY · PubMed Central
Fig. 3 Distribution of documents by Year
Fig. 3 Distribution of documents by YearCC BY · PubMed Central
Fig. 4 Distribution of articles by location
Fig. 4 Distribution of articles by locationCC BY · PubMed Central
Fig. 5 Distribution of publications using method
Fig. 5 Distribution of publications using methodCC BY · PubMed Central
Fig. 6 Document distribution by source
Fig. 6 Document distribution by sourceCC BY · PubMed Central

Verification

Claims the paper makes

10 Valid · 18 Not valid · 11 Not checked · 74 Own results and interpretations

52 more claims · 11 with their cited sources checked

Create a free account to see all 113 claims and to verify the 4 references still open.

See all claims — free account
  • It may state two separate claims as one. 48
  • The quote only partly states this claim. 4
  • Probably not a substantial claim. 20

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 43 references

Research works printed but not linked: 4

  1. Eysenbach G. The Role of ChatGPT, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation With ChatGPT and a Call for Papers. JMIR Med Educ. 2023;9(1):e46885. https://doi.org/10.2196/46885. PubMed PMID: 36863937. · DOI · Paper page
  2. Yokose M, Hirosawa T, Sakamoto T, Kawamura R, Suzuki Y, Harada Y, et al. The Validity of Generative Artificial Intelligence in Evaluating Medical Students in Objective Structured Clinical Examination: Experimental Study. JMIR Form Res. 2025;9:e79465. https://doi.org/10.2196/79465. PubMed PMID: 41343812. · DOI · Paper page
  3. (PDF). Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review. Available from: https://www.researchgate.net/publication/390805456_Potential_Societal_Biases_of_ChatGPT_in_Higher_Education_A_Scoping_Review. Cited 3 Mar 2026.
  4. Alkhaaldi SMI, Kassab CH, Dimassi Z, Alsoud LO, Al Fahim M, Al Hageh C et al. Medical Student Experiences and Perceptions of ChatGPT and Artificial Intelligence: Cross-Sectional Study. JMIR Med Educ. 2023;9(1). https://doi.org/10.2196/51302. PubMed PMID: 38133911. · DOI · Paper page

Cited by

See all 1

Metadata

Authors
Ndacyayisenga, J. C. Y., Kidega, C., & Aciro Can, L.
Venue
BMC Medical Education
Published
2026-04-29
Volume
26
Issue
1
ISSN
1472-6920 (ISSN-L)
OpenAlex
W7157762857
Cited by
1
References
43
Citations by year
2026: 1
FWCI
0.61
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Education (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Education › Innovations in Medical Education
  • Social Sciences › Education › Artificial Intelligence in Education
Keywords
generative AI, assessment equity, non-native English speakers, medical students, automated grading, assessment validity, systematic bias, construct-irrelevant variance, artificial intelligence in medical education, bias auditing, student attitudes toward AI
MeSH terms
Diversity, Equity, Inclusion, Generative Artificial Intelligence, Educational Measurement, Humans, Students, Medical, Multilingualism

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The impact of generative artificial intelligence tools on assessment equity for non-native English-speaking medical students: a systematic review. https://app.cochat.ai/papers/10.1186/s12909-026-09303-7
The paper
Ndacyayisenga, J. C. Y., Kidega, C., & Aciro Can, L. (2026). The impact of generative artificial intelligence tools on assessment equity for non-native English-speaking medical students: a systematic review. BMC Medical Education. https://doi.org/10.1186/s12909-026-09303-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.