Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index; 29 of the 60 sources its claims cite were checked for support.

Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings

Open access

Abstract

The rapid adoption of generative artificial intelligence tools such as ChatGPT in educational settings has generated both enthusiasm and concern regarding their influence on student learning. While several studies report improvements in efficiency, confidence, and perceived understanding, evidence for durable conceptual learning and knowledge transfer remains mixed. This article examines these tensions through the concept of fluency illusion, a cognitive phenomenon in which information that is easy to process is mistakenly judged as being well understood. Using a narrative conceptual review approach, this study synthesizes findings from 41 publications identified through searches of Google Scholar, Scopus, Web of Science, and ERIC covering the period from 2022 to early 2026. The reviewed literature includes 28 empirical studies, nine conceptual or theoretical analyses, and four review articles addressing the use of ChatGPT in educational contexts. Across domains such as writing and language learning, STEM problem solving, feedback and tutoring, and assessment, the literature shows a recurring pattern in which fluent AI-generated responses increase learners’ confidence without consistently improving deeper conceptual understanding. Drawing on research from cognitive psychology and metacognition, this review proposes an integrative conceptual account of how fluent AI output may shape learners’ judgments of understanding and influence their engagement with learning tasks. The paper concludes by discussing implications for instructional design, assessment practices, and metacognitive scaffolding, and outlines directions for future research aimed at empirically examining the proposed framework and identifying strategies to reduce fluency-driven misjudgments while preserving the potential benefits of generative AI in education.

Overview

CoChat Summary

TL;DR

This narrative conceptual review synthesizes 41 publications on ChatGPT in education and proposes “fluency illusion” as a unifying explanation for why fluent AI output can raise confidence without consistently improving deep learning. It argues that learners may misattribute the ease and coherence of AI-generated responses to their own understanding, with implications for pedagogy, feedback, and assessment.

Key findings

  • The review corpus comprised 41 publications: 28 empirical studies, 9 conceptual/theoretical papers, and 4 review articles; studies were distributed across writing/language learning (17), STEM problem solving (11), feedback/tutoring (7), and assessment/academic integrity (6).
  • Across contexts, the literature shows a recurring pattern: fluent AI-generated responses increase learners’ confidence and perceived understanding, but evidence for durable conceptual learning and transfer remains mixed or limited.
  • In writing and language learning, students report benefits for brainstorming, drafting, paraphrasing, and linguistic accuracy, yet increased confidence does not consistently translate into improvements in argumentation quality, rhetorical control, or independent writing performance.
  • In STEM, feedback, and assessment contexts, students may follow AI explanations or accept fluent feedback uncritically, perform adequately on routine tasks, and struggle with novel or conceptually demanding problems; fluent AI output can also make it harder to distinguish independent understanding from tool-assisted performance.

Methods

  • Narrative conceptual review with a targeted literature search in Google Scholar, Scopus, Web of Science, and ERIC, covering January 2022 to January 2026; inclusion required educational contexts and ChatGPT or closely related generative AI, while purely technical model-development studies and non-educational contexts were excluded.
  • Screening involved duplicate removal, title/abstract review, and full-text assessment; methodological quality was considered using principles from CASP and MMAT, but no formal scoring system was applied. Synthesis was thematic narrative, grouping studies by writing, language learning, STEM, feedback, and assessment.
  • A descriptive bibliometric overview used the Dimensions database with a keyword search on titles/abstracts for 2022–2025 English articles in education/learning-related Fields of Research; the initial search retrieved 112 publications, and the final review corpus was 41 studies.

Limitations

  • Stated: the review is not exhaustive or fully systematic, does not quantify effect sizes or rank interventions, and its conclusions should be read as analytically grounded rather than statistically definitive; the proposed framework is a theoretical synthesis, not an empirically validated causal model.
  • Stated: the empirical literature on generative AI in education remains recent, methodologically diverse, and uneven; many cognitive mechanisms discussed require further empirical investigation, and current research relies heavily on short-term/cross-sectional designs and self-reported measures.
  • Evident: the bibliometric overview relies on a single database (Dimensions) and may underrepresent non-English or less-indexed journals; no formal quality scoring or meta-analytic synthesis was performed.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/info17030299

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

19 Valid · 16 Not valid · 26 Not checked · 94 Own results and interpretations

87 more claims · 11 with their cited sources checked

Create a free account to see all 155 claims and to verify the 6 references still open.

See all claims — free account
  • It may state two separate claims as one. 34
  • Probably not a substantial claim. 45

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

Show all 49 references

Research works printed but not linked: 7

  1. Anthonio, T. (2024). Linguistically-Informed Modelling of Potentials for Misunderstanding. [Ph.D. Thesis, Universität Stuttgart].
  2. Zhang, M., and Osman, K. (2025). Exploring the Implementation and Challenges of the ChatGPT in STEM Higher Education: A Systematic Literature Review. 2024 Yearbook Emerging Technologies in Learning, Springer.
  3. Dunlosky, J., and Metcalfe, J. (2008). Metacognition, Sage Publications.
  4. Bjork, 2011, Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning, Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society, Volume 2, 56
  5. Cianferoni, F. (2025). AI in Higher Education: From Literature to a Course-Anchored Chatbot. [Ph.D. Thesis, Politecnico di Torino].
  6. Topolinski, S. (2013). The sources of fluency: Identifying the underlying mechanisms of fluency effects. The Experience of Thinking, Psychology Press.
  7. Brown, M., Nic Giolla Mhichíl, M., Beirne, E., and Costello, E. (2020). Aligning Professional Identity with Institutional Culture: The Role of Educators’ Digital Fluency in Harnessing the Potential of Online and Technology Enhanced Learning. Proceedings of the 2019 ICDE World Conference on Online Learning, Dublin City University.

Cited by

See all 11

Metadata

Authors
Kumar, S., Mikayelyan, A., & Vorfolomeyeva, O.
Affiliations
  • American University of Armenia (AM)
  • Mahambet Otemiusly West Kazakhstan University (KZ)
Corresponding author
Kumar, Sachin
Venue
Information
Publisher
MDPI AG
Published
2026-03-19
Volume
17
Issue
3
Pages
299
ISSN
2078-2489 (ISSN-L)
OpenAlex
W7138833636
Cited by
11
References
49
Citations by year
2026: 11
FWCI
90.92
Type
journal article
Language
en
Open access
Locations
  • Information (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ChatGPT, generative AI in education, perceived understanding, metacognition, conceptual understanding, student self-confidence, AI-generated feedback, AI tutoring, assessment, metacognitive scaffolding, instructional design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings. https://app.cochat.ai/papers/10.3390/info17030299
The paper
Kumar, S., Mikayelyan, A., & Vorfolomeyeva, O. (2026). Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings. Information. https://doi.org/10.3390/info17030299

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.