CoChat
Sign Up
Cited by 55Teaching and Learning ProgrammingArtificial Intelligence in EducationSoftware Engineering Research

The Good and Bad of AI Tools in Novice Programming Education

Open access
Zviel-Girshin, Rina·2024·Education Sciences
DOI 10.3390/educsci14101089
Open this paper in CoChat — free accountOpen PDF

Abstract

As AI coding tools become more prevalent in programming, it is essential to understand how they influence programming education. This study, conducted in a first-semester Introduction to Programming course, aimed to determine the positive and negative effects of these tools on students’ learning experiences and their ability to develop essential programming skills. Using a mixed-methods approach, we collected data from 73 teams of engineering students over a 12-week period. Students completed surveys and reported on their AI tool usage. We analyzed this data quantitatively to identify trends in tool familiarity, usage, and student satisfaction. Additionally, qualitative analysis of student reports provided insights into the specific ways AI tools were used and their perceived benefits and drawbacks. The findings revealed a significant increase in AI tool familiarity (from 28% to 100%) and usage among students. Students’ satisfaction with AI tools improved over time. The most prevalent tasks for which novice programmers used AI tools included creating comments (91.7%), identifying and correcting bugs (80.2%), and seeking information (68.5%), while other tasks were less common. While these tools offered benefits like assisting in learning and enhancing real-world relevance, they also raised concerns about cheating, over-reliance on AI tools, and a limited understanding of core programming concepts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/educsci14101089

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Gen AI can serve as an effective scaffold for learners without formal programming background.

    “Their work suggests that Gen AI can serve as an effective scaffold for learners without formal programming background, though it also raises questions about dependency and skill transfer.”

    From Democratizing Data Analysis: Can Generative AI Tools Enable Non-STEM Students to Complete Complex Coding Tasks with Minimal Training? (2025) · cites this paper as Zviel et al. (2024) [2]

    Section 4.2 (‘The Good,’ page unavailable) reports that AI tools enhanced novice programmers’ learning experiences through feedback, information retrieval, and help with bugs, supporting the suggestion that they can scaffold learning. Section 4.1 (‘General Discussion,’ page unavailable) also discusses dependency and independent problem-solving. But Section 2.1 identifies participants as first-semester novice programmers; it does not establish that they had no formal programming background. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Excessive dependence on GenAI tools may undermine the development of algorithmic thinking, error detection, and analytical reasoning.

    “While research has demonstrated that GenAI can support learning and foster creativity, excessive dependence on these tools may undermine the development of algorithmic thinking, error detection, and analytical reasoning (Kazemitabaar et al., 2023;Mogavi et al., 2024;Zviel-Girshin, 2024).”

    From Integrating Generative AI into Programming Education: Student Perceptions and the Challenge of Correcting AI Errors (2025) · cites this paper as Zviel-Girshin, 2024

    Section 4.3 (‘The Bad’; page not provided) says excessive reliance can weaken independent problem-solving and understanding of programming principles, leaving students more vulnerable to algorithmic errors. Section 3.5 reports unsuccessful bug identification and fixing, but the paper does not establish that dependence undermines the development of error-detection skills specifically. Section 6 notes that critical-thinking skills were not directly measured. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • These mixed views reflect broader debates in the literature on the dual nature of AI in education.

    “These mixed views reflect broader debates in the literature on the dual nature of AI in education -simultaneously enhancing productivity and creativity while raising concerns about over-dependence and diminished cognitive engagement (Mogavi et al., 2024;Usher, 2025;Zviel-Girshin, 2024).”

    From Integrating Generative AI into Programming Education: Student Perceptions and the Challenge of Correcting AI Errors (2025) · cites this paper as Zviel-Girshin, 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. Becker, B.A.; Denny, P.; Finnie-Ansley, J.; Luxton-Reilly, A.; Prather, J.; Santos, E.A. Programming is hard-or at least it used to be: Educational opportunities and challenges of ai code generation. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1; Association for Computing Machinery: New York, NY, USA, 2023; pp. 500-506.
  2. Bommasani, R.; Hudson, D.A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M.S.; Bohg, J.; Bosselut, A.; Brunskill, E.; et al. On the opportunities and risks of foundation models. arXiv 2021, arXiv:2108.07258. · arXiv
  3. Peng, S.; Kalliamvakou, E.; Cihon, P.; Demirer, M. The impact of ai on developer productivity: Evidence from github copilot. arXiv 2023, arXiv:2302.06590. · arXiv
  4. Yilmaz, R.; Yilmaz, F.G.K. The effect of generative artificial intelligence (AI)-based tool use on students' computational thinking skills, programming self-efficacy and motivation. Comput. Educ. Artif. Intell. 2023, 4, 100147. [CrossRef] · DOI · Paper page
  5. Lau, S.; Guo, P. From "Ban it till we understand it" to "Resistance is futile": How university programming instructors plan to adapt as more students use AI code generation and explanation tools such as ChatGPT and GitHub Copilot. In Proceedings of the 2023 ACM Conference on International Computing Education Research-Volume 1; Association for Computing Machinery: New York, NY, USA, 2023; pp. 106-121.
  6. Yin, J.; Goh, T.T.; Yang, B.; Xiaobin, Y. Conversation technology with micro-learning: The impact of chatbot-based learning on students' learning motivation and performance. J. Educ. Comput. Res. 2021, 59, 154-177. [CrossRef] · DOI · Paper page
  7. Biswas, S. Role of ChatGPT in Computer Programming. Mesopotamian J. Comput. Sci. 2023, 2023, 9-15. [CrossRef] [PubMed] · DOI
  8. Haleem, A.; Javaid, M.; Singh, R.P. An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Trans. Benchmarks Stand. Eval. 2022, 2, 100089. [CrossRef] · DOI · Paper page
  9. Vukojičić, M.; Krstić, J. ChatGPT in programming education: ChatGPT as a programming assistant. InspirED Teach. Voice 2023, 2023, 7-13.
  10. Covill, A.E. College students' perceptions of the traditional lecture method. Coll. Stud. J. 2011, 45, 92-102.
  11. Yue, S. The Evolution of Pedagogical Theory: From Traditional to Modern Approaches and Their Impact on Student Engagement and Success. J. Educ. Educ. Res. 2024, 7, 226-230. [CrossRef] · DOI
  12. Stukalenko, N.M.; Zhakhina, B.B.; Kukubaeva, A.K.; Smagulova, N.K.; Kazhibaeva, G.K. Studying innovation technologies in modern education. Int. J. Environ. Sci. Educ. 2016, 11, 7297-7308.
  13. Pardos, Z.A.; Bhandari, S. Learning gain differences between ChatGPT and human tutor generated algebra hints. arXiv 2023, arXiv:2302.06871. · arXiv
  14. Chen, R.; Zhao, H. ChatGPT in Creative Writing Courses in Chinese Universities: Application and Research. In Proceedings of the 2024 12th International Conference on Information and Education Technology (ICIET), Yamaguchi, Japan, 18-20 March 2024; pp. 243-247.
  15. Fischer, R.; Luczak-Roesch, M.; Karl, J.A. What does chatgpt return about human values? exploring value bias in chatgpt using a descriptive value theory. arXiv 2023, arXiv:2304.03612. · arXiv
  16. Huang, Z.; Mao, Y.; Zhang, J. The Influence of Artificial Intelligence Technology on College Students' Learning Effectiveness from the Perspective of Constructivism-Taking ChatGPT as an Example. J. Educ. Humanit. Soc. Sci. 2024, 30, 40-46. [CrossRef] · DOI
  17. Koć-Januchta, M.M.; Schönborn, K.J.; Roehrig, C.; Chaudhri, V.K.; Tibell, L.A.; Heller, H.C. "Connecting concepts helps put main ideas together": Cognitive load and usability in learning biology with an AI-enriched textbook. Int. J. Educ. Technol. High. Educ. 2022, 19, 11. [CrossRef] · DOI
  18. Sandoval-Medina, C.; Arévalo-Mercado, C.A.; Muñoz-Andrade, E.L.; Muñoz-Arteaga, J. Self-Explanation Effect of Cognitive Load Theory in Teaching Basic Programming. J. Inf. Syst. Educ. 2024, 35, 303-312. [CrossRef] · DOI
  19. Mandai, K.; Tan, M.J.H.; Padhi, S.; Pang, K.T. A Cross-Era Discourse on ChatGPT's Influence in Higher Education through the Lens of John Dewey and Benjamin Bloom. Educ. Sci. 2024, 14, 614. [CrossRef] · DOI
  20. Kuhail, M.A.; Mathew, S.S.; Khalil, A.; Berengueres, J.; Shah, S.J.H. "Will I be replaced?" Assessing ChatGPT's effect on software development and programmer perceptions of AI tools. Sci. Comput. Program. 2024, 235, 103111. [CrossRef] · DOI · Paper page
  21. Dick, M.; Sheard, J.; Bareiss, C.; Carter, J.; Joyce, D.; Harding, T.; Laxer, C. Addressing student cheating: Definitions and solutions. ACM SigCSE Bull. 2002, 35, 172-184. [CrossRef] · DOI
  22. Sheard, J.; Simon Butler, M.; Falkner, K.; Morgan, M.; Weerasinghe, A. Strategies for maintaining academic integrity in first-year computing courses. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education, Bologna, Italy, 3-5 July 2017; pp. 244-249.
  23. Albluwi, I. Plagiarism in programming assessments: A systematic review. ACM Trans. Comput. Educ. (TOCE) 2019, 20, 1-28. [CrossRef] · DOI · Paper page
  24. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; de Oliveira Pinto, H.P.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. Evaluating large language models trained on code. arXiv 2021, arXiv:2107.03374. · arXiv

Cited by

Sorted by citations

Authors
Zviel-Girshin, R.
Affiliations
Ruppin Academic Center (IL)
Corresponding author
Zviel-Girshin, Rina
ORCID
Zviel-Girshin, Rina
Venue
Education Sciences
Publisher
MDPI AG
Published
2024-10-06
Volume
14
Issue
10
Pages
1089
ISSN
2227-7102 (ISSN-L)
DOI
10.3390/educsci14101089
OpenAlex
W4403192978
Cited by
55
References
19
Citations by year
2026: 32 · 2025: 23
FWCI
20.79
Type
journal article
Language
en
Open access
Licence
Locations
  • Education Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
Ruppin Academic Center (33139)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
AI coding tools, programming for beginners, introductory programming education, student satisfaction, overreliance on AI, programming skills development, engineering students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review