Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 11 of the 30 sources its claims cite were checked for support.

The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches

Open access
Fan, Guangrui, Liu, Dandan, Zhang, Rui, Pan, Lihu · 2025 · International Journal of STEM Education

Abstract

This study investigates the impact of AI-assisted pair programming on undergraduate students’ intrinsic motivation, programming anxiety, and performance, relative to both human–human pair programming and individual programming approaches. A quasi-experimental design was conducted over two academic years (2023–2024) with 234 undergraduate students in a Java web application development course. Intact class sections were randomly assigned to AI-assisted pair programming (using GPT-3.5 Turbo in 2023 and Claude 3 Opus in 2024), human–human pair programming, or individual programming conditions. Data on intrinsic motivation, programming anxiety, collaborative perceptions, and programming performance were collected at three time points using validated instruments. Compared to individual programming, AI-assisted pair programming significantly increased intrinsic motivation ( p < .001, d = 0.35) and reduced programming anxiety ( p < .001), producing outcomes comparable to human–human pair programming. AI-assisted groups also outperformed both individual and human–human groups in programming tasks ( p < .001). However, human–human pair programming fostered the highest perceptions of collaboration and social presence, surpassing both AI-assisted and individual conditions ( p < .001). Mediation analysis revealed that perceived usefulness of the AI assistant significantly mediated the relationship between the programming approach and student outcomes, highlighting the importance of positive perceptions in leveraging AI tools for educational benefits. No significant differences emerged between the two AI models employed, indicating that both GPT-3.5 Turbo and Claude 3 Opus provided similar benefits. While AI-assisted pair programming enhances motivation, reduces anxiety, and improves performance, it does not fully match the collaborative depth and social presence achieved through human–human pairing. These findings highlight the complementary strengths of AI and human interaction: AI support can bolster learning outcomes, yet human partners offer richer social engagement. As AI capabilities advance, educators should integrate such tools thoughtfully, ensuring that technology complements rather than replaces the interpersonal dynamics and skill development central to effective programming education.

Overview

CoChat Summary

TL;DR

This quasi-experiment with 234 undergraduates found AI-assisted pair programming improved intrinsic motivation, reduced programming anxiety, and raised programming performance relative to individual programming, with outcomes comparable to human–human pairing. However, human–human pairs still reported the strongest collaboration and social presence, and GPT-3.5 Turbo and Claude 3 Opus did not differ significantly.

Key findings

  • Compared with individual programming, AI-assisted pair programming significantly increased intrinsic motivation (p < .001, d = 0.35) and reduced programming anxiety (p < .001), with outcomes comparable to human–human pair programming.
  • At T3, the average AI-assisted group had higher intrinsic motivation (M = 5.532, SD = 0.819) than human–human pairs (M = 5.423, SD = 0.852) and individual programmers (M = 5.237, SD = 1.147); it also had lower anxiety (M = 2.355, SD = 0.692) than human–human pairs (M = 2.643, SD = 0.775) and individual programmers (M = 2.756, SD = 0.741).
  • AI-assisted groups showed superior T3 performance (M = 83.901, SD = 10.611) versus human–human pairs (M = 80.123, SD = 11.654) and individual programmers (M = 75.789, SD = 11.234); the abstract reports AI-assisted groups outperformed both other groups (p < .001), while the results note the AI vs. human–human post hoc difference was not statistically significant but a separate comparison was significant (t = 2.456, p = 0.015, d = 0.355).
  • Human–human pair programming produced the highest collaboration and social presence: MANOVA showed significant group effects on CLS, F(3, 230) = 23.487, p < .001, η²p = 0.234, and SPQ, F(3, 230) = 18.923, p < .001, η²p = 0.198; GPT-3.5 Turbo and Claude 3 Opus did not differ significantly on any measured variable.

Methods

  • Quasi-experimental design over two academic years (2023–2024) with 234 undergraduate students in a Java web application development course at Taiyuan University of Science and Technology; six intact class sections were cluster-randomized to AI-assisted pair programming (GPT-3.5 Turbo in 2023, Claude 3 Opus in 2024), human–human pair programming, or individual programming.
  • Measures included the Intrinsic Motivation Inventory, Programming Anxiety Scale, Collaborative Learning Scale, Social Presence Questionnaire, adapted Technology Acceptance Model survey, and programming performance metrics (assignment grades, rubric-based code quality, error rates); data were collected at T1, T2, and T3.
  • Analyses included mixed between-within subjects ANOVA, MANOVA with discriminant function analysis, SEM-based mediation with perceived usefulness as mediator, and ANCOVA controlling for GPA.

Limitations

  • The paper states the quasi-experimental design may leave unaccounted confounding variables; findings may not generalize beyond the specific institution, educational level, and cultural context; self-report measures may be biased; performance measures may not capture all programming skill development; and the AI models studied were later superseded.
  • The paper also acknowledges that despite supervised labs, honor codes, and activity logs, students may have accessed unapproved external resources outside class, which could affect internal validity.
  • Evident from the text: outcomes are short-term within one course, and the abstract’s claim that AI-assisted groups outperformed human–human groups in programming tasks is not fully consistent with the results section’s post hoc statement that this difference was not statistically significant.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s40594-025-00537-3

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

6 Valid · 3 Not valid · 18 Not checked · 142 Own results and interpretations

89 more claims

Create a free account to see all 169 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 41
  • Probably not a substantial claim. 38
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(3, 230) = 0.072reported p = .975 · recomputed p = .975

    “F(3, 230) = 0.072, p = .975”

  • ConsistentF(2, 460) = 47.623reported p < .001 · recomputed p < .001

    “F(2, 460) = 47.623, p < .001”

  • ConsistentF(3, 230) = 3.842reported p = .01 · recomputed p = .010

    “F(3, 230) = 3.842, p = .010”

  • ConsistentF(6, 460) = 2.976reported p = .007 · recomputed p = .007

    “F(6, 460) = 2.976, p = .007”

  • ConsistentF(2, 460) = 53.189reported p < .001 · recomputed p < .001

    “F(2, 460) = 53.189, p < .001”

5 of 18 · the rest aren’t loaded

13 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 57 references

Research works printed but not linked: 4

  1. Govender, I., & Grayson, D. (2006). Learning to program and learning to teach programming: A closer look. EdMedia+ Innovate Learning (pp. 1687–1693). Association for the Advancement of Computing in Education.
  2. Gu, X., & Cai, H. (2021). Predicting the future of artificial intelligence and its educational impact: A thought experiment based on social science fiction. Educational Research, 42(05), 137–147.
  3. Jiao, J., Chen, L., & Wu, W. (2023). Educational issues triggered by ChatGPT: Possible impacts and counter measures. Chinese Journal of ICT in Education, 29(03), 19–32.
  4. Nunnally, J., & Bernstein, I. (1994). Psychometric Theory (3rd ed.). MacGraw-Hill.

Cited by

See all 110

Metadata

Authors
Fan, G., Liu, D., Zhang, R., & Pan, L.
Affiliations
  • Taiyuan University of Science and Technology (CN)
  • University of Malaya (MY)
Corresponding author
Fan, Guangrui
Venue
International Journal of STEM Education
Published
2025-03-04
Volume
12
Issue
1
ISSN
2196-7822 (ISSN-L)
OpenAlex
W4408153468
Cited by
109
References
57
Citations by year
2026: 69 · 2025: 40
FWCI
96.99
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of STEM Education (publishedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Humanities and Social Science Fund of Ministry of Education of China (23YJCZH299)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
pair programming, intrinsic motivation, cooperative learning, programming performance, GPT-3.5-turbo, Claude 3 Opus, social presence, undergraduate programming education, Java web application development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches. https://app.cochat.ai/papers/10.1186/s40594-025-00537-3
The paper
Fan, G., Liu, D., Zhang, R., & Pan, L. (2025). The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches. International Journal of STEM Education. https://doi.org/10.1186/s40594-025-00537-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.