Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 3 of the 18 sources its claims cite were checked for support.

College students' metacognitive awareness of generative-AI reliance: an experimental study of decision confidence and attribution bias

Open access
Ren, Shaowei · 2026 · Frontiers in Psychology

Abstract

Introduction: Generative AI is increasingly used by college students for explanation, evaluation, and decision support, raising the question of whether perceived reliance accurately tracks the extent to which AI advice shapes final judgments. Methods: In a three-condition between-subjects experiment, 342 undergraduate students completed reasoning and information-evaluation tasks under independent decision-making, open multi-turn ChatGPT support, or the same ChatGPT support with a brief metacognitive reflection prompt. Participants reported initial and final answers and confidence, and AI-assisted chat logs were coded for the final recommendation, its correctness, and behavioral reliance. Results: Open ChatGPT support increased final confidence and was associated with 62.4% acceptance of incorrect AI advice. Reflection reduced incorrect-advice acceptance to 39.7% (OR = 0.40, 95% CI [0.28, 0.56], p < 0.001), improved awareness calibration (0.59 vs. 0.41, p < 0.001), and reduced the attribution-bias index (0.21 vs. 0.42, p = 0.002). Recommendation accuracy was comparable between the two AI-assisted groups. Discussion: Brief metacognitive reflection improved immediate discrimination between useful and misleading AI advice without producing blanket rejection of AI. The exploratory association between calibration and overreliance is not interpreted as causal mediation, and the findings concern bounded academic decision tasks rather than long-term learning.

Overview

CoChat Summary

## TL;DR

This randomized experiment found that open multi-turn ChatGPT support increased college students’ confidence, but it also increased acceptance of incorrect AI advice and self-serving attribution bias. A brief metacognitive reflection prompt reduced incorrect-advice acceptance and improved alignment between perceived and behavioral reliance.

## Key findings

  • In the final sample of 342 undergraduate students (after excluding 18 participants), open ChatGPT support increased final confidence relative to independent decision-making: Mdiff = 7.7, 95% CI [4.37, 11.03], adjusted p < 0.001; reflection also raised confidence vs. independent decision-making, but less strongly (Mdiff = 3.9, 95% CI [0.57, 7.23], adjusted p = 0.044).
  • Open ChatGPT support led to 62.4% acceptance of incorrect AI advice, versus 39.7% with reflection; the effect was significant (OR = 0.40, 95% CI [0.28, 0.56], p < 0.001).
  • Awareness calibration was better under reflection: 0.59 vs. 0.41 (reported as higher = better), p < 0.001; the signed discrepancy shifted from M = -0.22 under open ChatGPT to M = -0.02 with reflection.
  • The AI-specific attribution-bias index was lower with reflection: 0.21 vs. 0.42 under open ChatGPT, p = 0.002; source-level analysis also found higher AI-advice attribution in the open ChatGPT condition by 0.46 points on the 1–7 scale (p = 0.004).

## Methods

  • Design: Three-condition between-subjects experiment: independent decision-making, open multi-turn ChatGPT support, and the same ChatGPT support plus a brief metacognitive reflection prompt.
  • Sample/data: 342 undergraduate students from Anhui University of Science and Technology; they completed reasoning and information-evaluation tasks online in spring 2026. The task set included 24 tasks across analytical reasoning, data interpretation, and information credibility evaluation.
  • Analysis: Trial-level outcomes were analyzed with mixed-effects models with participant and task random intercepts; participant-level outcomes used linear models. The study also used logistic mixed-effects models for incorrect-advice acceptance, plus robustness checks, covariate-adjusted models, and an exploratory bootstrap indirect-effect analysis.

## Limitations

  • The paper states that the sample came from one university, so the findings may not generalize to other institutions, cultures, disciplines, or age groups.
  • The tasks were relatively short, so the results concern immediate decision behavior rather than semester-long learning, durable skill acquisition, or transfer beyond the task.
  • The paper notes that the exploration of awareness calibration is not temporally or operationally independent of the behavioral outcome, so the exploratory indirect-effect analysis cannot establish causal mediation.
  • Evident from the design: the ChatGPT interaction was bounded by a standardized starter prompt, a maximum of two follow-up questions (three turns total), and a controlled online environment, which limits ecological breadth.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2026.1926110

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

2 Valid · 4 Not valid · 8 Not checked · 136 Own results and interpretations

47 more claims

Create a free account to see all 150 claims and to verify the 15 references still open.

See all claims — free account
  • Probably not a substantial claim. 60
  • It may state two separate claims as one. 64
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = 0.57reported p = .57 · recomputed p = .569

    “z = 0.57, p = .57”

  • Consistentt(224.8) = 2.2reported p = .029 · recomputed p = .029

    “t (224.8) = 2.20, p = .029”

  • Consistentt(222.94) = 3.1reported p = .002 · recomputed p = .002

    “t (222.94) = 3.10, p = .002”

  • Consistentχ²(6) = 22.1reported p = .001 · recomputed p = .001

    “χ 2 (6) = 22.1, p = .001”

4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Citations

Cites

Show all 33 references

Research works printed but not linked: 2

  1. Bandura, 1997, Self-Efficacy: The Exercise of Control
  2. Dunlosky, 2009, Metacognition

Metadata

Authors
Ren, S.
Affiliations
Anhui University of Science and Technology (CN)
Corresponding author
Ren, Shaowei
Venue
Frontiers in Psychology
Published
2026-09-15
Volume
17
Pages
1926110-1926110
ISSN
1664-1078 (ISSN-L)
OpenAlex
W7213299655
Cited by
0
References
33
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
metacognitive awareness, metacognitive reflection, decision confidence, attribution bias, AI-assisted decision-making, ChatGPT, overreliance on AI, calibration, university students, information evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: College students' metacognitive awareness of generative-AI reliance: an experimental study of decision confidence and attribution bias. https://app.cochat.ai/papers/10.3389/fpsyg.2026.1926110
The paper
Ren, S. (2026). College students' metacognitive awareness of generative-AI reliance: an experimental study of decision confidence and attribution bias. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2026.1926110

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.