Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings
Open accessThe DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index; 29 of the 60 sources its claims cite were checked for support.
Abstract
The rapid adoption of generative artificial intelligence tools such as ChatGPT in educational settings has generated both enthusiasm and concern regarding their influence on student learning. While several studies report improvements in efficiency, confidence, and perceived understanding, evidence for durable conceptual learning and knowledge transfer remains mixed. This article examines these tensions through the concept of fluency illusion, a cognitive phenomenon in which information that is easy to process is mistakenly judged as being well understood. Using a narrative conceptual review approach, this study synthesizes findings from 41 publications identified through searches of Google Scholar, Scopus, Web of Science, and ERIC covering the period from 2022 to early 2026. The reviewed literature includes 28 empirical studies, nine conceptual or theoretical analyses, and four review articles addressing the use of ChatGPT in educational contexts. Across domains such as writing and language learning, STEM problem solving, feedback and tutoring, and assessment, the literature shows a recurring pattern in which fluent AI-generated responses increase learners’ confidence without consistently improving deeper conceptual understanding. Drawing on research from cognitive psychology and metacognition, this review proposes an integrative conceptual account of how fluent AI output may shape learners’ judgments of understanding and influence their engagement with learning tasks. The paper concludes by discussing implications for instructional design, assessment practices, and metacognitive scaffolding, and outlines directions for future research aimed at empirically examining the proposed framework and identifying strategies to reduce fluency-driven misjudgments while preserving the potential benefits of generative AI in education.
Overview
CoChat Summary
TL;DR
This narrative conceptual review synthesizes 41 publications on ChatGPT in education and proposes “fluency illusion” as a unifying explanation for why fluent AI output can raise confidence without consistently improving deep learning. It argues that learners may misattribute the ease and coherence of AI-generated responses to their own understanding, with implications for pedagogy, feedback, and assessment.
Key findings
- The review corpus comprised 41 publications: 28 empirical studies, 9 conceptual/theoretical papers, and 4 review articles; studies were distributed across writing/language learning (17), STEM problem solving (11), feedback/tutoring (7), and assessment/academic integrity (6).
- Across contexts, the literature shows a recurring pattern: fluent AI-generated responses increase learners’ confidence and perceived understanding, but evidence for durable conceptual learning and transfer remains mixed or limited.
- In writing and language learning, students report benefits for brainstorming, drafting, paraphrasing, and linguistic accuracy, yet increased confidence does not consistently translate into improvements in argumentation quality, rhetorical control, or independent writing performance.
- In STEM, feedback, and assessment contexts, students may follow AI explanations or accept fluent feedback uncritically, perform adequately on routine tasks, and struggle with novel or conceptually demanding problems; fluent AI output can also make it harder to distinguish independent understanding from tool-assisted performance.
Methods
- Narrative conceptual review with a targeted literature search in Google Scholar, Scopus, Web of Science, and ERIC, covering January 2022 to January 2026; inclusion required educational contexts and ChatGPT or closely related generative AI, while purely technical model-development studies and non-educational contexts were excluded.
- Screening involved duplicate removal, title/abstract review, and full-text assessment; methodological quality was considered using principles from CASP and MMAT, but no formal scoring system was applied. Synthesis was thematic narrative, grouping studies by writing, language learning, STEM, feedback, and assessment.
- A descriptive bibliometric overview used the Dimensions database with a keyword search on titles/abstracts for 2022–2025 English articles in education/learning-related Fields of Research; the initial search retrieved 112 publications, and the final review corpus was 41 studies.
Limitations
- Stated: the review is not exhaustive or fully systematic, does not quantify effect sizes or rank interventions, and its conclusions should be read as analytically grounded rather than statistically definitive; the proposed framework is a theoretical synthesis, not an empirically validated causal model.
- Stated: the empirical literature on generative AI in education remains recent, methodologically diverse, and uneven; many cognitive mechanisms discussed require further empirical investigation, and current research relies heavily on short-term/cross-sectional designs and self-reported measures.
- Evident: the bibliometric overview relies on a single database (Dimensions) and may underrepresent non-English or less-indexed journals; no formal quality scoring or meta-analytic synthesis was performed.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
19 Valid · 16 Not valid · 26 Not checked · 94 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly, research in STEM learning contexts suggests that students may successfully follow AI-generated explanations yet struggle to reproduce or adapt the same reasoning independently in new problems [13,14].”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 1. Introduction
What the sources say
- Not matchedcites [13] Zhang, M., and Osman, K. (2025). Exploring the Implementation and Challenges of the ChatGPT in STEM Higher Education: A Systematic Literature Review. 2024 Yearbook Emerging Technologies in Learning, Springer.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [14] Zhai, 2024, The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: A systematic review, Smart Learn. Environ. sourceExperimental — accuracy not yet measured
The Introduction says that overdependence on AI may diminish students’ independent thought and analytical reasoning, which supports the general concern. It does not describe STEM learners following AI-generated explanations and then failing to reproduce or adapt the reasoning on new problems; searches for STEM and reasoning transfer did not identify that specific finding. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“From a cognitive perspective, this phenomenon aligns closely with what has been described as fluency illusion [7].”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 1. Introduction
What the sources say
- Partly supportedcites [7] Heersmink, 2024, A phenomenology and epistemology of large language models: Transparency, trust, and trustworthiness, Ethics Inf. Technol. sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says that LLM responses can seem to show understanding and that clear, confident, articulate output can lead users to place undue trust in them. This supports the proposed resemblance, but our copy does not name or define the phenomenon as a “fluency illusion.” (LLM, conf=0.78)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The fluency of AI-generated text frequently creates an impression of correctness, depth, and understanding that may not correspond to genuine learning [4].”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 1. Introduction
What the sources say
- Partly supportedcites [4] Mekheimer, 2025, Generative AI-assisted feedback and EFL writing: A study on proficiency, revision frequency and writing quality, Discov. Educ. sourceExperimental — accuracy not yet measured
Section 6.3 (page unavailable) says that passively accepting AI feedback without processing it could hinder deeper learning, and the Conclusion reports no significant improvement in writing content despite gains in surface-level accuracy and organization. The paper does not say that fluent AI-generated text creates a misleading impression of correctness, depth, or understanding; it studies Grammarly primarily as a feedback tool, not a text generator. (LLM, conf=0.76)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The conversational nature of large language models further amplifies this effect by fostering a sense of dialogue and understanding [31].”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 3.4. Surface Coherence and Deep Learning
What the sources say
- Supportedcites [31] Grifoni, 2025, The bewitching AI: The Illusion of Communication with Large Language Models, Philos. Technol. source
Section 7, “How the Bewitchment Arises,” says that LLMs’ plausible conversational responses make people perceive the system as taking the interaction into account and think communication is taking place. This supports the claim that conversational interaction fosters a sense of dialogue and understanding. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Learners with stronger prior knowledge or more developed metacognitive regulation may be better able to recognize the limitations of fluent AI-generated explanations and therefore less likely to equate processing ease with genuine understanding [33].”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 3.4. Surface Coherence and Deep Learning
What the sources say
- Could not verifycites [33] Murayama, 2016, When enough is not enough: Information overload and metacognitive decisions to stop studying information, J. Exp. Psychol. Learn. Mem. Cogn. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
87 more claims · 11 with their cited sources checked
Create a free account to see all 155 claims and to verify the 6 references still open.
- It may state two separate claims as one. 34
- Probably not a substantial claim. 45
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > Abstract
“this study synthesizes findings from 41 publications identified through searches of Google Scholar, Scopus, Web of Science, and ERIC”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 8. Descriptive Bibliometric Overview
“The initial search retrieved 112 publications.After screening titles and abstracts according to the inclusion and exclusion criteria described in Section 2, duplicates and non-relevant records were removed, resulting in a final review corpus of 41 studies.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 8. Descriptive Bibliometric Overview
“The search query used in the database was: ("ChatGPT" OR "generative AI" OR "large language model") AND (education OR learning OR classroom OR teaching).”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 9. Conclusions
“| Core keywords | ("ChatGPT" OR "generative AI" OR "large language model") AND (education OR learning OR classroom) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 4. ChatGPT in Classroom Settings: What the Literature Shows
“Table 1 summarizes recurring applications, benefits, concerns, and associated fluency-related patterns observed across classroom contexts.”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 9. Conclusions
“Table: 1 Table 1 . Key Gaps and Future Research Directions in AI-Mediated Learning.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 9. Conclusions
“limited records indexed in early 2026 are included because the database search was conducted in January 2026[64].”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 9. Conclusions
“| Publication years | 2022-2025 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > Data availability
“Data sharing is not applicable to this article as no data were created or analyzed in this study.”
§ Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings > 8. Descriptive Bibliometric Overview
“The Dimensions database [64] was used to retrieve publication metadata using a keyword-based search applied to titles and abstracts.”
Citations
Cites
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Cognitive Architecture and Instructional Design: 20 Years Later
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09465-5 · Cited by 2,337
Zhai, C., Wibowo, S., & Li, L. D. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00316-7 · Cited by 1,567
Handbook of Metacognition in Education
Hacker, D. J., Dunlosky, J., & Graesser, A. C. (2009) · doi:10.4324/9780203876428 · Cited by 1,509
Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning
Koriat, A. (1997) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.126.4.349 · Cited by 1,363
The misunderstood limits of folk science: An illusion of explanatory depth
Rozenblit, L., & Keil, F. (2002) · Cognitive Science · doi:10.1207/s15516709cog2605_1 · Cited by 1,032
Credibility and trust of information in online environments: The use of cognitive heuristics
Metzger, M. J., & Flanagin, A. J. (2013) · Journal of Pragmatics · doi:10.1016/j.pragma.2013.07.012 · Cited by 1,000
ChatGPT and large language models in academia: opportunities and challenges
Meyer, J. G., Urbanowicz, R. J., Martin, P. C. N., O’Connor, K., Li, R., Peng, P.-C., Bright, T. J., Tatonetti, N., Won, K. J., Gonzalez-Hernandez, G., et al. (2023) · BioData Mining · doi:10.1186/s13040-023-00339-9 · Cited by 569
Research works printed but not linked: 7
- Anthonio, T. (2024). Linguistically-Informed Modelling of Potentials for Misunderstanding. [Ph.D. Thesis, Universität Stuttgart].
- Zhang, M., and Osman, K. (2025). Exploring the Implementation and Challenges of the ChatGPT in STEM Higher Education: A Systematic Literature Review. 2024 Yearbook Emerging Technologies in Learning, Springer.
- Dunlosky, J., and Metcalfe, J. (2008). Metacognition, Sage Publications.
- Bjork, 2011, Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning, Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society, Volume 2, 56
- Cianferoni, F. (2025). AI in Higher Education: From Literature to a Course-Anchored Chatbot. [Ph.D. Thesis, Politecnico di Torino].
- Topolinski, S. (2013). The sources of fluency: Identifying the underlying mechanisms of fluency effects. The Experience of Thinking, Psychology Press.
- Brown, M., Nic Giolla Mhichíl, M., Beirne, E., and Costello, E. (2020). Aligning Professional Identity with Institutional Culture: The Role of Educators’ Digital Fluency in Harnessing the Potential of Online and Technology Enhanced Learning. Proceedings of the 2019 ICDE World Conference on Online Learning, Dublin City University.
Cited by
Algorithmic Authority Bias in AI-Assisted Learning: An Integrative Cognitive Framework
Sony, S. D., & Mus, K. (2026) · International Educational Review · doi:10.58693/ier.415 · Cited by 2
“Recent studies have shown that students using LLMs often believe they understand topics more deeply than they actually do (Kumar et al., 2026; Belghith et al., 2024).”
Darko, E., Zhan-yong, Q., & Wang, Z. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1829056 · Cited by 3
“This issue also mirrors wider international worries about the accuracy and dependability of GenAI outputs, as discussed in other research (Kumar et al., 2026).”
A Metacognitive Blind Spot: Student Comprehension, AI Reliance, and the Conceptual Difficulty Gap
Crk, I., & Gultepe, E. (2026) · Education Sciences · doi:10.3390/educsci16071102
Lv, T., Bao, W., Li, S., Sun, C., Chen, S., & Zhang, M. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1914167 · Cited by 1
Sivasubramanian, G. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1958667
Tan, S. F., Chan, J. I. L., Chew, C. M., & Eak, A. D. (2026) · ASEAN Journal of Open and Distance Learning · doi:10.64233/xzya2444
Kani, M., & Handayani, F. (2026) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss71466.2026.11715470
“In addition, the “fluency illusion” effect [37]. suggests that information appearing clear, convincing, and easy to understand is more likely to be perceived as credible without careful verification.”
Influential
Metadata
- Authors
- Kumar, S., Mikayelyan, A., & Vorfolomeyeva, O.
- Affiliations
- American University of Armenia (AM)
- Mahambet Otemiusly West Kazakhstan University (KZ)
- Corresponding author
- Kumar, Sachin
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2026-03-19
- Volume
- 17
- Issue
- 3
- Pages
- 299
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W7138833636
- Semantic Scholar
- 32f45a61eb223dbddd6e9fd7d19df42f4012e636
- Cited by
- 11
- References
- 49
- Citations by year
- 2026: 11
- FWCI
- 90.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- ChatGPT, generative AI in education, perceived understanding, metacognition, conceptual understanding, student self-confidence, AI-generated feedback, AI tutoring, assessment, metacognitive scaffolding, instructional design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.