The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 3 of the 18 sources its claims cite were checked for support.
College students' metacognitive awareness of generative-AI reliance: an experimental study of decision confidence and attribution bias
Open accessAbstract
Introduction: Generative AI is increasingly used by college students for explanation, evaluation, and decision support, raising the question of whether perceived reliance accurately tracks the extent to which AI advice shapes final judgments. Methods: In a three-condition between-subjects experiment, 342 undergraduate students completed reasoning and information-evaluation tasks under independent decision-making, open multi-turn ChatGPT support, or the same ChatGPT support with a brief metacognitive reflection prompt. Participants reported initial and final answers and confidence, and AI-assisted chat logs were coded for the final recommendation, its correctness, and behavioral reliance. Results: Open ChatGPT support increased final confidence and was associated with 62.4% acceptance of incorrect AI advice. Reflection reduced incorrect-advice acceptance to 39.7% (OR = 0.40, 95% CI [0.28, 0.56], p < 0.001), improved awareness calibration (0.59 vs. 0.41, p < 0.001), and reduced the attribution-bias index (0.21 vs. 0.42, p = 0.002). Recommendation accuracy was comparable between the two AI-assisted groups. Discussion: Brief metacognitive reflection improved immediate discrimination between useful and misleading AI advice without producing blanket rejection of AI. The exploratory association between calibration and overreliance is not interpreted as causal mediation, and the findings concern bounded academic decision tasks rather than long-term learning.
Overview
CoChat Summary
## TL;DR
This randomized experiment found that open multi-turn ChatGPT support increased college students’ confidence, but it also increased acceptance of incorrect AI advice and self-serving attribution bias. A brief metacognitive reflection prompt reduced incorrect-advice acceptance and improved alignment between perceived and behavioral reliance.
## Key findings
- In the final sample of 342 undergraduate students (after excluding 18 participants), open ChatGPT support increased final confidence relative to independent decision-making: Mdiff = 7.7, 95% CI [4.37, 11.03], adjusted p < 0.001; reflection also raised confidence vs. independent decision-making, but less strongly (Mdiff = 3.9, 95% CI [0.57, 7.23], adjusted p = 0.044).
- Open ChatGPT support led to 62.4% acceptance of incorrect AI advice, versus 39.7% with reflection; the effect was significant (OR = 0.40, 95% CI [0.28, 0.56], p < 0.001).
- Awareness calibration was better under reflection: 0.59 vs. 0.41 (reported as higher = better), p < 0.001; the signed discrepancy shifted from M = -0.22 under open ChatGPT to M = -0.02 with reflection.
- The AI-specific attribution-bias index was lower with reflection: 0.21 vs. 0.42 under open ChatGPT, p = 0.002; source-level analysis also found higher AI-advice attribution in the open ChatGPT condition by 0.46 points on the 1–7 scale (p = 0.004).
## Methods
- Design: Three-condition between-subjects experiment: independent decision-making, open multi-turn ChatGPT support, and the same ChatGPT support plus a brief metacognitive reflection prompt.
- Sample/data: 342 undergraduate students from Anhui University of Science and Technology; they completed reasoning and information-evaluation tasks online in spring 2026. The task set included 24 tasks across analytical reasoning, data interpretation, and information credibility evaluation.
- Analysis: Trial-level outcomes were analyzed with mixed-effects models with participant and task random intercepts; participant-level outcomes used linear models. The study also used logistic mixed-effects models for incorrect-advice acceptance, plus robustness checks, covariate-adjusted models, and an exploratory bootstrap indirect-effect analysis.
## Limitations
- The paper states that the sample came from one university, so the findings may not generalize to other institutions, cultures, disciplines, or age groups.
- The tasks were relatively short, so the results concern immediate decision behavior rather than semester-long learning, durable skill acquisition, or transfer beyond the task.
- The paper notes that the exploration of awareness calibration is not temporally or operationally independent of the behavioral outcome, so the exploratory indirect-effect analysis cannot establish causal mediation.
- Evident from the design: the ChatGPT interaction was bounded by a standardized starter prompt, a maximum of two follow-up questions (three turns total), and a controlled online environment, which limits ecological breadth.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 4 Not valid · 8 Not checked · 136 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Reflection reduced incorrectadvice acceptance to 39.7% (OR = 0.40, 95% CI [0.28, 0.56], p < 0.001)”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“was associated with 62.4% acceptance of incorrect AI advice”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“improved awareness calibration (0.59 vs. 0.41, p < 0.001)”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Open ChatGPT support increased final confidence”
- Also stated in: § 4.3 Descriptive statistics · plain
“Open ChatGPT support increased final confidence relative to independent decision-making.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.3 Descriptive statistics · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Behavioral reliance was positively associated with final confidence and overreliance.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
47 more claims
Create a free account to see all 150 claims and to verify the 15 references still open.
- Probably not a substantial claim. 60
- It may state two separate claims as one. 64
- The quote only partly states this claim. 1
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 0.57reported p = .57 · recomputed p = .569
“z = 0.57, p = .57”
- Consistentt(224.8) = 2.2reported p = .029 · recomputed p = .029
“t (224.8) = 2.20, p = .029”
- Consistentt(222.94) = 3.1reported p = .002 · recomputed p = .002
“t (222.94) = 3.10, p = .002”
- Consistentχ²(6) = 22.1reported p = .001 · recomputed p = .001
“χ 2 (6) = 22.1, p = .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Telling more than we can know: Verbal reports on mental processes.
Nisbett, R. E., & Wilson, T. D. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.231 · Cited by 11,317
Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry
Flavell, J. H. (1979) · American Psychologist · doi:10.1037/0003-066x.34.10.906 · Cited by 11,282 · cited 2×
“Metacognition refers to people’s knowledge and regulation of their own cognition, including their ability to monitor uncertainty, evaluate evidence, and adjust strategies when knowledge is incomplete (Flavell, 1979; Nelson and Narens, 1990; Koriat, 1997).”
An attributional theory of achievement motivation and emotion
Weiner, B. (1985) · Psychological Review · doi:10.1037/0033-295x.92.4.548 · Cited by 7,846 · cited 1×
“Attribution theory explains how individuals interpret the causes of success and failure (Weiner, 1985).”
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731 · cited 1×
“Studies of university students also indicate a paradoxical pattern in which generative-AI use can increase perceived efficiency or self-efficacy while simultaneously strengthening technological dependence or false self-efficacy (Zhang and Xu, 2025; Jia et al., 2025; Kruger and Dunning, 1999).”
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“On the positive side, generative AI may support individualized tutoring, rapid feedback, language learning, and access to explanations for students with diverse learning needs (Kasneci et al., 2023; Tlili et al., 2023; Shahzad et al., 2025).”
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 3×
“Automation bias refers to excessive weighting of automated recommendations, especially when users accept erroneous advice (Parasuraman and Riley, 1997; Parasuraman and Manzey, 2010).”
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“Research on algorithm appreciation and algorithm aversion further suggests that people may either defer to algorithms or reject them depending on prior experience, perceived expertise, and error visibility (Dietvorst et al., 2015; Logg et al., 2019).”
Assessing Metacognitive Awareness
Schraw, G., & Dennison, R. S. (1994) · Contemporary Educational Psychology · doi:10.1006/ceps.1994.1033 · Cited by 3,199 · cited 1×
“Metacognition includes both knowledge about cognition and regulation of cognition (Flavell, 1979; Dunlosky and Metcalfe, 2009; Schraw and Dennison, 1994).”
Metamemory: A Theoretical Framework and New Findings
Nelson, T. O. (1990) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60053-5 · Cited by 2,583 · cited 2×
“Metacognition refers to people’s knowledge and regulation of their own cognition, including their ability to monitor uncertainty, evaluate evidence, and adjust strategies when knowledge is incomplete (Flavell, 1979; Nelson and Narens, 1990; Koriat, 1997).”
Research works printed but not linked: 2
- Bandura, 1997, Self-Efficacy: The Exercise of Control
- Dunlosky, 2009, Metacognition
Metadata
- Authors
- Ren, S.
- Affiliations
- Anhui University of Science and Technology (CN)
- Corresponding author
- Ren, Shaowei
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2026-09-15
- Volume
- 17
- Pages
- 1926110-1926110
- ISSN
- 1664-1078 (ISSN-L)
- PMC
- PMC13619835
- OpenAlex
- W7213299655
- Cited by
- 0
- References
- 33
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Psychology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- metacognitive awareness, metacognitive reflection, decision confidence, attribution bias, AI-assisted decision-making, ChatGPT, overreliance on AI, calibration, university students, information evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: College students' metacognitive awareness of generative-AI reliance: an experimental study of decision confidence and attribution bias. https://app.cochat.ai/papers/10.3389/fpsyg.2026.1926110
- The paper
- Ren, S. (2026). College students' metacognitive awareness of generative-AI reliance: an experimental study of decision confidence and attribution bias. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2026.1926110
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.