Students’ Experiences of Using ChatGPT in an Undergraduate Programming Course
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Increasing use of artificial intelligence tools in programming education calls for a deeper understanding of their effect on students’ learning. This paper presents a study that investigates the experiences of part-time undergraduate students using ChatGPT in a five-week Java programming course. After each exercise, students provided feedback via anonymous surveys in which they rated different suitability aspects of ChatGPT. The majority viewed ChatGPT positively and suitable for learning programming concepts. However, its suitability for specific implementation tasks received mixed reviews. Students found it easy to adapt ChatGPT’s generated code to the exercises’ implementation tasks. The students primarily used it for acquiring background knowledge, learning syntax and programming concepts and suggesting suitable algorithms. Yet, some abstained from using it due to concerns to not garner sufficient programming proficiency, retrieving partially incorrect or misleading generated code, preferring an independent working style, or general skepticism about its benefits. Finally, in response to our findings, we also discuss three perspective directions for improving the suitability of LLM chatbots for students in programming education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090 · cited 1×
“Chatbots are computer programs that mimic human-like conversation with users using natural language or text, thus creating a sense of interpersonal interaction [1], [2].”
BackgroundChatGPT for Education and Research: Opportunities, Threats, and Strategies
Rahman, M. M., & Watanobe, Y. (2023) · Applied Sciences · doi:10.3390/app13095783 · Cited by 1,080 · cited 3×
“Itcanbe summarized that only two studies [20], [21] incorporated students’ experiences about using ChatGPT in programming education.”
BackgroundChatbots applications in education: A systematic review
Okonkwo, C. W., & Ade-Ibijola, A. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100033 · Cited by 879
Chatbots for learning: A review of educational chatbots for the Facebook Messenger
Smutny, P., & Schreiberova, P. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103862 · Cited by 770 · cited 1×
“However, educational chatbots face a range of constraints and obstacles [14], [15].”
BackgroundChen, Y., Jensen, S., Albert, L. J., Gupta, S., & Lee, T. (2023) · Information Systems Frontiers · doi:10.1007/s10796-022-10291-4 · Cited by 685 · cited 1×
“As part of their applications, they support the learning process [6], [7], enhance student engagement [8], [9], evaluate students’ progress [10], [11], and perform administrative tasks [12], [13].”
BackgroundEssel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-022-00362-6 · Cited by 428 · cited 1×
“As part of their applications, they support the learning process [6], [7], enhance student engagement [8], [9], evaluate students’ progress [10], [11], and perform administrative tasks [12], [13].”
BackgroundStudying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580919 · Cited by 366 · cited 2×
“Itcanbe summarized that only two studies [20], [21] incorporated students’ experiences about using ChatGPT in programming education.”
BackgroundMethodA Survey on Conversational Agents/Chatbots Classification and Design Techniques
Hussain, S., Ameri Sianaki, O., & Ababneh, N. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-15035-8_93 · Cited by 279
Using grounded theory to study the experience of software development
Adolph, S., Hall, W., & Kruchten, P. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9152-6 · Cited by 235
A Tool for Introducing Computer Science with Automatic Formative Assessment
Benotti, L., Martinez, M. C., & Schapachnik, F. (2018) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2682084 · Cited by 141 · cited 1×
“As part of their applications, they support the learning process [6], [7], enhance student engagement [8], [9], evaluate students’ progress [10], [11], and perform administrative tasks [12], [13].”
BackgroundSavelka, J., Agarwal, A., Bogart, C., Song, Y., & Sakr, M. (2023) · Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 · doi:10.1145/3587102.3588792 · Cited by 109 · cited 1×
“Savelka et al. [26] examined the ability of generative pre-trained transformers, specifically text-davinci-003 , to pass programming assessments in introductory and intermediate Python courses.”
BackgroundSáiz-Manzanares, M. C., Marticorena-Sánchez, R., Martín-Antón, L. J., González Díez, I., & Almeida, L. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e12843 · Cited by 96 · cited 1×
“As part of their applications, they support the learning process [6], [7], enhance student engagement [8], [9], evaluate students’ progress [10], [11], and perform administrative tasks [12], [13].”
BackgroundBull, C., & Kharrufa, A. (2024) · IEEE Software · doi:10.1109/ms.2023.3300574 · Cited by 82 · cited 2×
“A study conducted by Bull and Kharrufa [23] explored current practices, challenges, and opinions regarding generative AI tools in industrial software development through exploratory interviews with software professionals.”
Background
Cited by
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Wang, J., & Fan, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04787-y · Cited by 322
“…leveraging ChatGPT to assist students in academic writing (Nugroho et al. 2024; Song and Song, 2023), facilitate the comprehension of complex knowledge concepts (Haindl and Weinberger, 2024; Sun et al. 2024; Zhou and Kim, 2024), and solve complex problems (Küchemann et al. 2023; Urban et al. 2024).”
RetractedGenerative AI in education and research: A systematic mapping review
Yusuf, A., Pervin, N., Román‐González, M., & Noor, N. M. (2024) · Review of Education · doi:10.1002/rev3.3489 · Cited by 182
Ursavaş, Ö. F., Yalçın, Y., İslamoğlu, H., Bakır-Yalçın, E., & Cukurova, M. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00535-z · Cited by 69
“For example, in computer science and engineering, students used ChatGPT for tasks such as debugging, summarizing topics, learning programming concepts, and suggesting algorithms (Ada, 2024; Haindl & Weinberger, 2024).”
Does ChatGPT Help Novice Programmers Write Better Code? Results From Static Code Analysis
Haindl, P., & Weinberger, G. (2024) · IEEE Access · doi:10.1109/access.2024.3445432 · Cited by 45
“It complements our previous qualitative study regarding the suitability of ChatGPT for programming exercises and typical application scenarios from students’ perspective [8].”
ResultSelf-citation
Metadata
- Authors
- Haindl, P., & Weinberger, G.
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 43519-43529
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4393065879
- Cited by
- 96
- References
- 13
- FWCI
- 37.20
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.