Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index; 11 of the 24 sources its claims cite were checked for support.

Generative AI without guardrails can harm learning: Evidence from high school mathematics

Open access
Bastani, Hamsa, Bastani, Osbert, Sungu, Alp, Ge, Haosen, Kabakcı, Özge, Mariman, Rei · 2025 · Proceedings of the National Academy of Sciences

Abstract

Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning—namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface (“GPT Base”) and one with prompts designed to safeguard learning (“GPT Tutor”). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)—i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a “crutch” during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity.

Overview

CoChat Summary

TL;DR

In a high school math RCT, GPT-4 tutoring improved practice performance but the unguarded GPT Base version harmed later unassisted exam performance, while teacher-designed guardrails in GPT Tutor largely eliminated the harm. The results suggest generative AI can boost performance while undermining learning unless deployed with safeguards.

Key findings

  • On assisted practice problems, GPT Base and GPT Tutor raised normalized scores by 0.137 and 0.361 out of 1, respectively, versus a control mean of 0.284—increases of 48% and 127%.
  • On the subsequent unassisted exam, GPT Base reduced performance by 0.054 out of 1, or 17% relative to the control mean of 0.321; GPT Tutor’s estimate was −0.004 and statistically indistinguishable from control.
  • When asked “What is the answer?” across 57 practice problems, GPT Base gave a correct answer only 51% of the time, with logical errors 42% and arithmetic errors 8%; its errors hurt practice performance but did not significantly spill over to corresponding exam problems.
  • Students in GPT Base performed worse on the exam but did not perceive worse learning or performance, while GPT Tutor students perceived better performance despite no exam improvement.

Methods

  • Randomized controlled trial at a large high school in Turkey during Fall 2023–2024, covering about fifty 9th–11th-grade classes and nearly 1,000 students across four 90-min sessions; classrooms were assigned to control, GPT Base, or GPT Tutor.
  • Each session had teacher review, assisted practice with randomized AI access, and an unassisted closed-book exam; GPT Base was a standard GPT-4 chat tutor, while GPT Tutor used teacher-designed hints and problem-specific solutions/common mistakes to avoid giving answers.
  • Main analysis used intention-to-treat regressions on normalized practice and exam grades with previous GPA controls and session, grader, grade-level, and teacher fixed effects, with SEs clustered at the classroom level; additional analyses examined problem-level outcomes, GPT error rates, and NLP-clustered student messages.

Limitations

  • Stated by the paper: the study covers one topic (mathematics), one high school in Turkey, and Fall 2023, when generative AI was still new; it focuses on short-term outcomes, and mechanisms would benefit from controlled experiments with finer interventions and monitoring.
  • Stated by the paper: objective math evaluation may not generalize to subjects such as writing, and both closed and open-weight models have improved since the deployment.
  • Evident from the text: honors-designated classrooms were excluded from the main sample, five class sessions did not use the assigned treatment due to external circumstances, and GPT Tutor remained passive with no positive exam effect.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.2422633122

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Prompts used in GPT Base and GPT Tutor for the first 11th grade practice problem in the first session.
Fig. 1. Prompts used in GPT Base and GPT Tutor for the first 11th grade practice problem in the first session.CC BY-NC-ND · PubMed Central
Fig. 2. Fraction of times GPT Base returns an incorrect answer due to a logical (Left) or arithmetic (Right) error for 57 practice problems across grades and sessions. (Note that GPT Tutor does not return an answer by design, and is further given the correct answer in the prompt).
Fig. 2. Fraction of times GPT Base returns an incorrect answer due to a logical (Left) or arithmetic (Right) error for 57 practice problems across grades and sessions. (Note that GPT Tutor does not return an answer by design, and is further given the correct answer in the prompt).CC BY-NC-ND · PubMed Central
Fig. 3. Student engagement—given by (A) average number of student messages per problem, and (B) average fraction of student session conversations that have no superficial messages (simply restating the question or asking for the answer) per session—by treatment (GPT Base and GPT Tutor) over time.
Fig. 3. Student engagement—given by (A) average number of student messages per problem, and (B) average fraction of student session conversations that have no superficial messages (simply restating the question or asking for the answer) per session—by treatment (GPT Base and GPT Tutor) over time.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

8 Valid · 0 Not valid · 4 Not checked · 71 Own results and interpretations

33 more claims

Create a free account to see all 83 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 34
  • It may state two separate claims as one. 20

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

12 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 27 references

Research works printed but not linked: 1

  1. C. Terwiesch, “Would chat GPT get a Wharton MBA” in Mack Institute White Paper (Wharton School, University of Pennsylvania, 2023).

Cited by

See all 309

Metadata

Authors
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R.
Affiliations
  • University of Pennsylvania (US)
  • Wharton AI & Analytics
  • International Business School (HU)
  • Independent
Corresponding author
  • Bastani, Hamsa
  • Bastani, Osbert
  • Sungu, Alp
Venue
Proceedings of the National Academy of Sciences
Published
2025-06-25
Volume
122
Issue
26
Pages
e2422633122-e2422633122
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W4411627694
Cited by
309
References
27
Citations by year
2026: 282 · 2025: 23
FWCI
314.35
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
Funders
  • Wharton AI and Analytics Initiative (N/A)
  • Fishman Davidson Center at Wharton (N/A)
  • Wharton Global Initiatives (N/A)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
generative AI in education, AI tutors, high school mathematics, GPT-4, guardrails, tool design, skill acquisition, learning outcomes, overreliance on AI, student performance
MeSH terms
Adolescent, Artificial Intelligence, Female, Humans, Learning, Male, Mathematics, Problem Solving, Schools, Students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Generative AI without guardrails can harm learning: Evidence from high school mathematics. https://app.cochat.ai/papers/10.1073/pnas.2422633122
The paper
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2422633122

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.