ChatGPT in the Classroom. Exploring Its Potential and Limitations in a Functional Programming Course
Open accessAbstract
In November 2022, OpenAI has introduced ChatGPT – a chatbot based on supervised and reinforcement learning. Not only can it answer questions emulating human-like responses, but it can also generate code from scratch or complete coding templates provided by the user. ChatGPT can generate unique responses which render any traditional anti-plagiarism tool useless. Its release has ignited a heated debate about its usage in academia, especially by students. We have found, to our surprise, that our students at POLITEHNICA University of Bucharest (UPB) have been using generative AI tools (ChatGPT and its predecessors) for solving homework, for at least 6 months. We therefore set out to explore the capabilities of ChatGPT and assess its value for educational purposes. We used ChatGPT to solve all our coding assignments for the semester from our UPB Functional Programming course. We discovered that, although ChatGPT provides correct answers in 68% of the cases, only around half of those are legible solutions which can benefit students in some form. On the other hand, ChatGPT has a very good ability to perform code review on student programming homework. Based on these findings, we discuss the pros and cons of ChatGPT in a teaching environment, as well as means for integrating GPT models for generating code reviews, in order to improve the code-writing skills of students.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868 · see citing works
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · see citing works
A Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514 · see citing works
Mathematical Capabilities of ChatGPT
Frieder, S., Pinchetti, L., Chevalier, A., Griffiths, R.-R., Salvatori, T., Lukasiewicz, T., Petersen, P. C., & Berner, J. (2023) · arXiv · doi:10.48550/arxiv.2301.13867 · Cited by 385 · see citing works
Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580919 · Cited by 366 · cited 1× · see citing works
“The paper of Kazemitabaar et al. (2023) performs a study aimed at assessing how students improve when using Copilot.”
The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023) · arXiv · doi:10.48550/arxiv.2302.06590 · Cited by 287 · cited 1× · see citing works
“A more recent study conducted by Peng et al. (2023) involving 95 participants has yielded a very different outcome.”
ChatGPT and Software Testing Education: Promises & Perils
Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023) · 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw58534.2023.00078 · Cited by 270 · cited 10× · see citing works
“There is a great deal of lively debate surrounding its potential uses, impact on research: Stokel-Walker (2023), science: Frieder et al. (2023), and education: Lambeets (2023), Elsen-Rooney (2023), Jalil et al. (2023), Lau and Guo (2023).”
InfluentialUsing GitHub Copilot to Solve Simple Programming Problems
Wermelinger, M. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569830 · Cited by 191 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Batista, J., Mesquita, A., & Carnaz, G. (2024) · Information · doi:10.3390/info15110676 · Cited by 256 · see citing works
How Do Programming Students Use Generative AI?
Rahe, C., & Maalej, W. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715762 · Cited by 45 · see citing works
“Furthermore, reliance on code generation can negatively impact code authoring skills [Jošt et al. 2024; Kazemitabaar et al. 2023].”
InfluentialMatobobo, C., Ncube, P. D. N., Ngesimani, N. L., Dzvapatsva, G. P., & Chinhamo, E. (2025) · 2025 IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016317 · Cited by 17 · see citing works
Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation
Gardella, N., Pettit, R., & Riggs, S. L. (2024) · Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 · doi:10.1145/3649217.3653615 · Cited by 15 · see citing works
“Popovici reported a similar finding for assignments in a functional programming course [32] .”
Challenges and Limitations of Generative AI in Education
Gökoğlu, S. (2024) · Advances in Educational Technologies and Instructional Design · doi:10.4018/979-8-3693-1351-0.ch008 · Cited by 15 · see citing works
Zhang, L., Jiang, Q., Xiong, W., & Zhao, W. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251333874 · Cited by 15 · see citing works
Chang, C. I., Choi, W. C., & Choi, I. C. (2024) · Proceedings of the 2024 7th Artificial Intelligence and Cloud Computing Conference · doi:10.1145/3719384.3719439 · Cited by 11 · see citing works
“For instance, Popovici [21] found that although ChatGPT provided correct answers in 68% of the cases, only around half were legible solutions that could benefit students.”
Navigating the Ethical and Societal Impacts of Generative AI in Higher Computing Education
Mak, J., Nakatumba-Nabende, J., Clear, T., Clear, A., Abluwi, I., Andrei, O., Angeli, L., MacNeil, S., Oyelere, S. S., Hale Rattigan, M., et al. (2025) · Proceedings of the 2025 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3760545.3783975 · Cited by 12 · see citing works
- Authors
- Popovici, M.-D.
- Affiliations
- Universitatea Națională de Știință și Tehnologie Politehnica București (RO)
- Corresponding author
- Popovici, Matei-Dan
- Venue
- International Journal of Human–Computer Interaction
- Publisher
- Informa UK Limited
- Published
- 2023-10-17
- Volume
- 40
- Issue
- 22
- Pages
- 7743-7754
- ISSN
- 1044-7318 (ISSN-L)
- 1532-7590
- OpenAlex
- W4387730663
- Cited by
- 70
- References
- 9
- Citations by year
- 2026: 16 · 2025: 34 · 2024: 20
- FWCI
- 16.43
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- International Journal of Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- ChatGPT, generative AI in education, functional programming education, code generation, code review, code readability, plagiarism, AI-assisted learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
