The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index; 11 of the 24 sources its claims cite were checked for support.
Generative AI without guardrails can harm learning: Evidence from high school mathematics
Open accessAbstract
Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning—namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface (“GPT Base”) and one with prompts designed to safeguard learning (“GPT Tutor”). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)—i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a “crutch” during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity.
Overview
CoChat Summary
TL;DR
In a high school math RCT, GPT-4 tutoring improved practice performance but the unguarded GPT Base version harmed later unassisted exam performance, while teacher-designed guardrails in GPT Tutor largely eliminated the harm. The results suggest generative AI can boost performance while undermining learning unless deployed with safeguards.
Key findings
- On assisted practice problems, GPT Base and GPT Tutor raised normalized scores by 0.137 and 0.361 out of 1, respectively, versus a control mean of 0.284—increases of 48% and 127%.
- On the subsequent unassisted exam, GPT Base reduced performance by 0.054 out of 1, or 17% relative to the control mean of 0.321; GPT Tutor’s estimate was −0.004 and statistically indistinguishable from control.
- When asked “What is the answer?” across 57 practice problems, GPT Base gave a correct answer only 51% of the time, with logical errors 42% and arithmetic errors 8%; its errors hurt practice performance but did not significantly spill over to corresponding exam problems.
- Students in GPT Base performed worse on the exam but did not perceive worse learning or performance, while GPT Tutor students perceived better performance despite no exam improvement.
Methods
- Randomized controlled trial at a large high school in Turkey during Fall 2023–2024, covering about fifty 9th–11th-grade classes and nearly 1,000 students across four 90-min sessions; classrooms were assigned to control, GPT Base, or GPT Tutor.
- Each session had teacher review, assisted practice with randomized AI access, and an unassisted closed-book exam; GPT Base was a standard GPT-4 chat tutor, while GPT Tutor used teacher-designed hints and problem-specific solutions/common mistakes to avoid giving answers.
- Main analysis used intention-to-treat regressions on normalized practice and exam grades with previous GPA controls and session, grader, grade-level, and teacher fixed effects, with SEs clustered at the classroom level; additional analyses examined problem-level outcomes, GPT error rates, and NLP-clustered student messages.
Limitations
- Stated by the paper: the study covers one topic (mathematics), one high school in Turkey, and Fall 2023, when generative AI was still new; it focuses on short-term outcomes, and mechanisms would benefit from controlled experiments with finer interventions and monitoring.
- Stated by the paper: objective math evaluation may not generalize to subjects such as writing, and both closed and open-weight models have improved since the deployment.
- Evident from the text: honors-designated classrooms were excluded from the main sample, five class sessions did not use the assigned treatment due to external circumstances, and GPT Tutor remained passive with no positive exam effect.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
8 Valid · 0 Not valid · 4 Not checked · 71 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“GPT Base often makes mistakes on math problems (22).”
§ Generative AI without guardrails can harm learning: Evidence from high school mathematics > Potential Mechanism: Asking for Solutions > GPT Errors vs. Student Performance.
What the sources say
- Partly supportedcites [22] S. Frieder, Mathematical capabilities of chatGPT. Adv. Neural Inf. Process. Syst. 36, 13867 (2024). sourceExperimental — accuracy not yet measured
The Results and Conclusion report that the evaluated ChatGPT versions often made mathematical mistakes and did not consistently produce high-quality calculations or proofs. But the paper identifies the evaluated models as ChatGPT versions and GPT-4, not “GPT Base,” so it does not establish the claim for that specific model. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This mismatch between perceived and actual learning has been observed in other settings (20, 21).”
§ Generative AI without guardrails can harm learning: Evidence from high school mathematics > Main Results > Student Perception.
What the sources say
- Not matchedcites [20] E. L. Bjork, R. A. Bjork, “Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning” in Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society, M. A. Gernsbacher, R. W. Pew, L. M. Hough, J. R. Pomerantz, Eds. (Worth Publishers, 2011), pp. 56–64.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [21] L. Deslauriers, L. S. McCarty, K. Miller, K. Callaghan, G. Kestin, Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc. Natl. Acad. Sci. U.S.A. 116, 19251– 19257 (2019). 31484770 PMC6765278 source
The Results and Discussion report that students’ perceived learning could be anticorrelated with their actual learning in active versus passive physics classes. The same section also describes a laboratory study in which perceived retention differed despite equal actual recall, supporting the reference to other settings. (LLM, conf=0.92)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“it continues to suffer from hallucinations where it provides confident but factually incorrect responses (12).”
§ Generative AI without guardrails can harm learning: Evidence from high school mathematics > Abstract
What the sources say
- Partly supportedcites [12] J. Maynez, S. Narayan, B. Bohnet, R. McDonald, “ On faithfulness and factuality in abstractive summarization” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, J. Tetreault, Eds. (Association for Computational Linguistics, 2020), pp. 1906– 1919. sourceExperimental — accuracy not yet measured
The Abstract and Section 7 report that neural abstractive summarization systems remain prone to hallucinations, and Section 5.3 says over 90% of BERTS2S hallucinations are erroneous. The paper studies summarization rather than generative AI responses generally, and it does not establish that those responses are delivered confidently. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both types of GPT Base errors negatively impact practice problem performance for students in the GPT Base arm (i.e., the coefficients of “GPT Base × Logical Error Rate” and “GPT Base × Arithmetic Error Rate” are statistically significantly negative in the practice performance regression).”
§ Generative AI without guardrails can harm learning: Evidence from high school mathematics > Potential Mechanism: Asking for Solutions > GPT Errors vs. Student Performance.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both GPT Base and GPT Tutor reduced grade dispersion in the assisted practice sessions, matching prior findings that generative AI assistance reduces the “skill gap” by providing the largest benefits for the weakest students (7–9).”
§ Generative AI without guardrails can harm learning: Evidence from high school mathematics > Main Results > Grade Dispersion.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
33 more claims
Create a free account to see all 83 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 34
- It may state two separate claims as one. 20
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“From their study, Bastani et al. (2025) concluded that learners employing a guided ChatGPT did much better on a post-intervention assessment than learners who were unrestricted in using ChatGPT. This was the case even though the unrestricted group's immediate performance during practice was much better.”
From Effects of Partial vs. Full Cognitive Offloading with Generative AI on Learning Retention (2026) · cites this paper as Bastani et al., 2025
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The risk is particularly acute for students, whose academic habits and metacognitive schemes are still in formative stages (Bastani, et al., 2025)”
From The Double-Edged Sword of Automation: Impact of Generative AI Adoption on Student's Critical Thinking Skills in Higher Education Considering Cognitive Offloading as a Mediator (2026) · cites this paper as Bastani, et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In a randomized study of high school students, Bastani et al. found that access to an AI tutor improved immediate task performance but resulted in substantially worse outcomes on subsequent unaided assessments, approximately 17% lower than controls, with the largest effects observed among weaker students 25 .”
From AI-induced never-skilling in medical education (2026) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Steering learners toward augmentation raises subsequent test scores in four of the six experiments: by 0.18 SD in Bastani et al. (2025), 0.22 SD in Gallegos (2026), 0.78 SD in Hou et al. (2026), and 0.13 to 0.14 SD in Kreijkes et al. (2026).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Bastani et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Steering learners toward augmentation raises subsequent test scores in four of the six experiments: by 0.18 SD in Bastani et al. (2025), 0.22 SD in Gallegos (2026), 0.78 SD in Hou et al. (2026), and 0.13 to 0.14 SD in Kreijkes et al. (2026).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Bastani et al., 2025
Experimental — accuracy not yet measured
12 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
(1964) · SSRN Electronic Journal · Cited by 5,128
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023) · arXiv · doi:10.48550/arxiv.2303.10130 · Cited by 2,330
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150
Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821936116 · Cited by 1,371
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · arXiv · doi:10.48550/arxiv.2005.00661 · Cited by 1,185
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
Research works printed but not linked: 1
- C. Terwiesch, “Would chat GPT get a Wharton MBA” in Mack Institute White Paper (Wharton School, University of Pennsylvania, 2023).
Cited by
Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking
Kumar, H., Vincentius, J., Jordan, E., & Anderson, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714198 · Cited by 129
Math Education with Large Language Models: Peril or Promise?
Kumar, H., Rothschild, D. M., Goldstein, D. G., & Hofman, J. (2023) · doi:10.2139/ssrn.4641653 · Cited by 54
Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
Wang, R., Ribeiro, A., Robinson, C., Loeb, S., & Demszky, D. (2024) · doi:10.21203/rs.3.rs-5363154/v1 · Cited by 53
Experimental evidence of the effects of large language models versus web search on depth of learning
Melumad, S., & Yun, J. H. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf316 · Cited by 61
A generative AI teaching assistant for personalized learning in medical education
Thesen, T., & Park, S. H. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02022-1 · Cited by 63
“…incorporating Socratic tutoring methods, where the AI guides students through problems with targeted questions rather than direct answers, could transform the system from a passive answer service into an active learning partner that promotes deeper understanding and long-term retention 30 .”
Metadata
- Authors
- Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R.
- Affiliations
- University of Pennsylvania (US)
- Wharton AI & Analytics
- International Business School (HU)
- Independent
- Corresponding author
- Bastani, Hamsa
- Bastani, Osbert
- Sungu, Alp
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2025-06-25
- Volume
- 122
- Issue
- 26
- Pages
- e2422633122-e2422633122
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC12232635
- OpenAlex
- W4411627694
- Cited by
- 309
- References
- 27
- Citations by year
- 2026: 282 · 2025: 23
- FWCI
- 314.35
- Type
- journal article
- Language
- en
- Update notices
- correction (2025-08-20)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- Funders
- Wharton AI and Analytics Initiative (N/A)
- Fishman Davidson Center at Wharton (N/A)
- Wharton Global Initiatives (N/A)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- generative AI in education, AI tutors, high school mathematics, GPT-4, guardrails, tool design, skill acquisition, learning outcomes, overreliance on AI, student performance
- MeSH terms
- Adolescent, Artificial Intelligence, Female, Humans, Learning, Male, Mathematics, Problem Solving, Schools, Students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative AI without guardrails can harm learning: Evidence from high school mathematics. https://app.cochat.ai/papers/10.1073/pnas.2422633122
- The paper
- Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2422633122
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.