The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 11 of the 30 sources its claims cite were checked for support.
The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches
Open accessAbstract
This study investigates the impact of AI-assisted pair programming on undergraduate students’ intrinsic motivation, programming anxiety, and performance, relative to both human–human pair programming and individual programming approaches. A quasi-experimental design was conducted over two academic years (2023–2024) with 234 undergraduate students in a Java web application development course. Intact class sections were randomly assigned to AI-assisted pair programming (using GPT-3.5 Turbo in 2023 and Claude 3 Opus in 2024), human–human pair programming, or individual programming conditions. Data on intrinsic motivation, programming anxiety, collaborative perceptions, and programming performance were collected at three time points using validated instruments. Compared to individual programming, AI-assisted pair programming significantly increased intrinsic motivation ( p < .001, d = 0.35) and reduced programming anxiety ( p < .001), producing outcomes comparable to human–human pair programming. AI-assisted groups also outperformed both individual and human–human groups in programming tasks ( p < .001). However, human–human pair programming fostered the highest perceptions of collaboration and social presence, surpassing both AI-assisted and individual conditions ( p < .001). Mediation analysis revealed that perceived usefulness of the AI assistant significantly mediated the relationship between the programming approach and student outcomes, highlighting the importance of positive perceptions in leveraging AI tools for educational benefits. No significant differences emerged between the two AI models employed, indicating that both GPT-3.5 Turbo and Claude 3 Opus provided similar benefits. While AI-assisted pair programming enhances motivation, reduces anxiety, and improves performance, it does not fully match the collaborative depth and social presence achieved through human–human pairing. These findings highlight the complementary strengths of AI and human interaction: AI support can bolster learning outcomes, yet human partners offer richer social engagement. As AI capabilities advance, educators should integrate such tools thoughtfully, ensuring that technology complements rather than replaces the interpersonal dynamics and skill development central to effective programming education.
Overview
CoChat Summary
TL;DR
This quasi-experiment with 234 undergraduates found AI-assisted pair programming improved intrinsic motivation, reduced programming anxiety, and raised programming performance relative to individual programming, with outcomes comparable to human–human pairing. However, human–human pairs still reported the strongest collaboration and social presence, and GPT-3.5 Turbo and Claude 3 Opus did not differ significantly.
Key findings
- Compared with individual programming, AI-assisted pair programming significantly increased intrinsic motivation (p < .001, d = 0.35) and reduced programming anxiety (p < .001), with outcomes comparable to human–human pair programming.
- At T3, the average AI-assisted group had higher intrinsic motivation (M = 5.532, SD = 0.819) than human–human pairs (M = 5.423, SD = 0.852) and individual programmers (M = 5.237, SD = 1.147); it also had lower anxiety (M = 2.355, SD = 0.692) than human–human pairs (M = 2.643, SD = 0.775) and individual programmers (M = 2.756, SD = 0.741).
- AI-assisted groups showed superior T3 performance (M = 83.901, SD = 10.611) versus human–human pairs (M = 80.123, SD = 11.654) and individual programmers (M = 75.789, SD = 11.234); the abstract reports AI-assisted groups outperformed both other groups (p < .001), while the results note the AI vs. human–human post hoc difference was not statistically significant but a separate comparison was significant (t = 2.456, p = 0.015, d = 0.355).
- Human–human pair programming produced the highest collaboration and social presence: MANOVA showed significant group effects on CLS, F(3, 230) = 23.487, p < .001, η²p = 0.234, and SPQ, F(3, 230) = 18.923, p < .001, η²p = 0.198; GPT-3.5 Turbo and Claude 3 Opus did not differ significantly on any measured variable.
Methods
- Quasi-experimental design over two academic years (2023–2024) with 234 undergraduate students in a Java web application development course at Taiyuan University of Science and Technology; six intact class sections were cluster-randomized to AI-assisted pair programming (GPT-3.5 Turbo in 2023, Claude 3 Opus in 2024), human–human pair programming, or individual programming.
- Measures included the Intrinsic Motivation Inventory, Programming Anxiety Scale, Collaborative Learning Scale, Social Presence Questionnaire, adapted Technology Acceptance Model survey, and programming performance metrics (assignment grades, rubric-based code quality, error rates); data were collected at T1, T2, and T3.
- Analyses included mixed between-within subjects ANOVA, MANOVA with discriminant function analysis, SEM-based mediation with perceived usefulness as mediator, and ANCOVA controlling for GPA.
Limitations
- The paper states the quasi-experimental design may leave unaccounted confounding variables; findings may not generalize beyond the specific institution, educational level, and cultural context; self-report measures may be biased; performance measures may not capture all programming skill development; and the AI models studied were later superseded.
- The paper also acknowledges that despite supervised labs, honor codes, and activity logs, students may have accessed unapproved external resources outside class, which could affect internal validity.
- Evident from the text: outcomes are short-term within one course, and the abstract’s claim that AI-assisted groups outperformed human–human groups in programming tasks is not fully consistent with the results section’s post hoc statement that this difference was not statistically significant.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 3 Not valid · 18 Not checked · 142 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“over-reliance on such technologies may inadvertently hinder the development of deeper cognitive and problem-solving abilities.”
What the sources say
- Supportedcites Abd-Alrazaq et al., 2023 source
The Challenges > Overreliance section says generative AI tools may hinder the development of foundational skills, including critical thinking and problem-solving. It also warns that readily available answers could reduce students’ motivation to investigate independently and reach their own conclusions. This supports the claim’s hedged wording. (LLM, conf=0.93)
- Supportedcites Duhaylungsod and Chavez, 2023 source
The Conclusion states that students expressed concerns that overreliance on AI could hinder critical thinking and problem-solving abilities. This supports the citing sentence’s qualified claim that such harm may occur. (LLM, conf=0.94)
- Supportedcites Koos and Wachsmann, 2023 source
The Limitations of ChatGPT section explicitly says overreliance may weaken students’ critical thinking and problem-solving skills. It adds that depending too heavily on AI-generated content could hinder the development of skills such as analyzing information and forming logical arguments. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“as well as a significant Time × Group interaction, F(6, 460) = 4.321, p < 0.001, η2p = 0.053.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Compared to individual programming, AI-assisted pair programming significantly increased intrinsic motivation (p < .001, d = 0.35) and reduced programming anxiety (p < .001), producing outcomes comparable to human–human pair programming.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AI-assisted groups also outperformed both individual and human–human groups in programming tasks (p < .001).”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No significant differences emerged between the two AI models employed, indicating that both GPT-3.5 Turbo and Claude 3 Opus provided similar benefits.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
89 more claims
Create a free account to see all 169 claims and every source verdict.
- It may state two separate claims as one. 41
- Probably not a substantial claim. 38
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Over-reliance on impersonal tool interaction can also leave relatedness needs under-served relative to peer or instructor collaboration (Fan et al., 2025).”
From Tool, tutor, or crutch?: A grounded theory of cognitive scaffolding and offloading in AI-assisted programming education (2026) · cites this paper as Fan et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In a quasi-experimental study with 234 undergraduates, Fan et al. (2025) reported that an AI pair-programming partner significantly improved task completion accuracy and reduced coding anxiety relative to both solo work and human pair programming.”
From Tool, tutor, or crutch?: A grounded theory of cognitive scaffolding and offloading in AI-assisted programming education (2026) · cites this paper as Fan et al., 2025
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“These patterns offer a mechanism-level interpretation of the mixed but generally positive outcome literature (Fan et al., 2025; Yilmaz & Yilmaz, 2023; Lyu et al., 2024; Shihab et al., 2025; Xia et al., 2025; Qi et al., 2025) alongside warnings about shallow conceptual change (Jose et al., 2025; Bastani et al., 2024).”
From Tool, tutor, or crutch?: A grounded theory of cognitive scaffolding and offloading in AI-assisted programming education (2026) · cites this paper as Fan et al., 2025
The cited source says what the claim attributes to it.
In the citing paper
“Empirical evidence suggests that AI assistance can improve short-term performance.”
From Tool, tutor, or crutch?: A grounded theory of cognitive scaffolding and offloading in AI-assisted programming education (2026) · cites this paper as Fan et al., 2025
The cited source says what the claim attributes to it.
In the citing paper
“This offers a process-level account of how performance and engagement metrics can improve (Fan et al., 2025; Yilmaz & Yilmaz, 2023; Lyu et al., 2024; Qi et al., 2025) while markers of conceptual integration or transfer may lag (Jose et al., 2025; Bastani et al., 2024).”
From Tool, tutor, or crutch?: A grounded theory of cognitive scaffolding and offloading in AI-assisted programming education (2026) · cites this paper as Fan et al., 2025
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(3, 230) = 0.072reported p = .975 · recomputed p = .975
“F(3, 230) = 0.072, p = .975”
- ConsistentF(2, 460) = 47.623reported p < .001 · recomputed p < .001
“F(2, 460) = 47.623, p < .001”
- ConsistentF(3, 230) = 3.842reported p = .01 · recomputed p = .010
“F(3, 230) = 3.842, p = .010”
- ConsistentF(6, 460) = 2.976reported p = .007 · recomputed p = .007
“F(6, 460) = 2.976, p = .007”
- ConsistentF(2, 460) = 53.189reported p < .001 · recomputed p < .001
“F(2, 460) = 53.189, p < .001”
13 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Results
“Baseline measures revealed comparable levels of intrinsic motivation (IMI), programming anxiety (PAS), perceived collaboration (CLS), social presence (SPQ), and technology acceptance (TAM) across all groups.”
§ Method
“Both the CLS and SPQ were administered at T2 and T3 to capture the development of collaborative and social perceptions over the course of the study.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Results
“Baseline measures revealed comparable levels of intrinsic motivation (IMI), programming anxiety (PAS), perceived collaboration (CLS), social presence (SPQ), and technology acceptance (TAM) across all groups.”
§ Method
“The survey was administered at T2 and T3 to measure changes in students’ technology acceptance over time as they gained experience interacting with the AI programming partner.”
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“To address potential issues of multiple comparisons, we employed the false discovery rate correction method (Benjamini & Hochberg, 1995).”
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · cited 11×
“Surveys (IMI, PAS, CLS, SPQ, TAM) were administered electronically at T1, T2, and T3 using secure, IRB-approved platforms.”
Influential(2013) · Cited by 44,563
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955 · cited 3×
“The decision to use SEM-based mediation was guided by standard practice in educational research, which encourages the examination of indirect effects and underlying mechanisms (Hayes, 2017).”
InfluentialA Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies
Venkatesh, V., & Davis, F. D. (2000) · Management Science · doi:10.1287/mnsc.46.2.186.11926 · Cited by 23,066 · cited 1×
“The Technology Acceptance Model (TAM) (Davis, 1989; Venkatesh & Davis, 2000) provides a robust framework for examining how users come to accept and utilize new technologies.”
Technology Acceptance Model 3 and a Research Agenda on Interventions
Venkatesh, V., & Bala, H. (2008) · Decision Sciences · doi:10.1111/j.1540-5915.2008.00192.x · Cited by 8,159
Ryan, R. M., & Deci, E. L. (2020) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2020.101860 · Cited by 6,037 · cited 3×
“The findings of this study have significant implications for theoretical frameworks in educational technology and psychology, particularly Self-Determination Theory (SDT), Social Presence Theory, and the Technology Acceptance Model (TAM).”
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304 · cited 1×
“By including entire class sections, we aimed to maintain the ecological validity of the study and minimize disruption to the educational environment (Durlak & DuPre, 2008).”
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · cited 1×
“Additionally, concerns about data privacy and security can hinder adoption, as AI tools often require access to user data to function optimally (Cotton et al., 2024).”
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Research works printed but not linked: 4
- Govender, I., & Grayson, D. (2006). Learning to program and learning to teach programming: A closer look. EdMedia+ Innovate Learning (pp. 1687–1693). Association for the Advancement of Computing in Education.
- Gu, X., & Cai, H. (2021). Predicting the future of artificial intelligence and its educational impact: A thought experiment based on social science fiction. Educational Research, 42(05), 137–147.
- Jiao, J., Chen, L., & Wu, W. (2023). Educational issues triggered by ChatGPT: Possible impacts and counter measures. Chinese Journal of ICT in Education, 29(03), 19–32.
- Nunnally, J., & Bernstein, I. (1994). Psychometric Theory (3rd ed.). MacGraw-Hill.
Cited by
Gaitantzi, A., & Kazanidis, I. (2025) · Applied Sciences · doi:10.3390/app15073960 · Cited by 65
Chen, S.-Y., Chen, W., & Lai, C. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13758-4 · Cited by 39
“Similarly, Fan et al. ( 2025 ), in a study on GAI-assisted pair programming, found that AI not only helped identify errors but also reduced students’ anxiety, improved their strategic adaptability, and enhanced their sense of collaboration.”
Cheng, C.-C., Wang, J.-S., Zhaı, X., & Yang, Y. (2025) · International Journal of STEM Education · doi:10.1186/s40594-025-00574-y · Cited by 42
“Guided by feedback and reflection, they iteratively refine their work, deepening their understanding of AI and enhancing their problem-solving skills (Cheng & Yang, 2023; Fan et al., 2025; Halawa et al., 2024; Ng et al., 2024a; Taylor et al., 2024).”
Fock, A., & Siller, H.-S. (2025) · International Journal of STEM Education · doi:10.1186/s40594-025-00589-5 · Cited by 28
“…engaging in mathematical endeavors has the potential to cultivate virtues connected to Human Flourishing such as a sense of play, truth, beauty, and justice (Su & Jackson, 2020), and AI-enhanced learning in programming education can increase motivation and reduce anxiety (Fan et al., 2025).”
Cognitive Atrophy Paradox of AI–Human Interaction: From Cognitive Growth and Atrophy to Balance
Kabashkin, I. (2025) · Information · doi:10.3390/info16111009 · Cited by 38
Metadata
- Authors
- Fan, G., Liu, D., Zhang, R., & Pan, L.
- Affiliations
- Taiyuan University of Science and Technology (CN)
- University of Malaya (MY)
- Corresponding author
- Fan, Guangrui
- Venue
- International Journal of STEM Education
- Published
- 2025-03-04
- Volume
- 12
- Issue
- 1
- ISSN
- 2196-7822 (ISSN-L)
- OpenAlex
- W4408153468
- Cited by
- 109
- References
- 57
- Citations by year
- 2026: 69 · 2025: 40
- FWCI
- 96.99
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- International Journal of STEM Education (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Humanities and Social Science Fund of Ministry of Education of China (23YJCZH299)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- pair programming, intrinsic motivation, cooperative learning, programming performance, GPT-3.5-turbo, Claude 3 Opus, social presence, undergraduate programming education, Java web application development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches. https://app.cochat.ai/papers/10.1186/s40594-025-00537-3
- The paper
- Fan, G., Liu, D., Zhang, R., & Pan, L. (2025). The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches. International Journal of STEM Education. https://doi.org/10.1186/s40594-025-00537-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.