Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Does ChatGPT Help Novice Programmers Write Better Code? Results From Static Code Analysis

Open access
Haindl, Philipp, Weinberger, Gerald · 2024 · IEEE Access

Abstract

In the realm of AI-enhanced programming education, there is growing interest in using such tools to help students understand good coding principles. This study investigates the impact of ChatGPT on code quality among part-time undergraduate students in introductory Java programming courses, who lack prior Java experience. The source code of 16 students from the control group (without ChatGPT) and 22 students from the treatment group (with ChatGPT) who completed identical programming exercises focused on coding conventions was analyzed. Static code analysis tools assessed adherence to a common coding convention ruleset and calculated cyclomatic and cognitive complexity metrics. The comparative analysis shows that the ChatGPT-assisted group significantly improved code quality, with fewer rule violations and reduced cyclomatic and cognitive complexities. The treatment group adhered more closely to coding standards and produced less complex code. Violations primarily occurred in line length, final parameters, and the extensibility of object-oriented programming (OOP). These findings suggest that ChatGPT can be beneficial in programming education by helping students write cleaner, less complex code and adhere to coding conventions. However, the study’s limitations, such as the small sample size and novice status of participants, call for further research with larger, more diverse populations and different educational contexts.

Overview

Integrity signals

Registry

Crossref · 10.1109/access.2024.3445432

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 26 references

Research works printed but not linked: 9

  1. 2024, OpenAI
  2. Izu, Exploring CS1 students notions of code quality, 12
  3. 2024, Checkstyle—Checkstyle 10.14.1
  4. 2024, PMD
  5. 2024, Code Conventions for the Java Programming Language: Contents
  6. 2024, Checkstyle—Checks
  7. 2024, Checkstyle—Suns Java Style
  8. 2024, Checkstyle/src/main/resources/sun.xml At Master
  9. 2024, Code Quality Tool & Secure Analysis With SonarQube

Cited by

See all 45

Metadata

Authors
Haindl, P., & Weinberger, G.
Affiliations
University of Applied Sciences St Pölten (AT)
Venue
IEEE Access
Published
2024
Volume
12
Pages
114146-114156
ISSN
2169-3536 (ISSN-L)
OpenAlex
W4401687261
Cited by
45
References
26
Citations by year
2026: 25 · 2025: 20
FWCI
9.82
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • Preprints.org (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Social Sciences › Education › Artificial Intelligence in Education
Keywords
ChatGPT, novice programmers, code quality, static code analysis, coding conventions, cyclomatic complexity, cognitive complexity, Java programming, introductory programming courses, part-time undergraduate students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Does ChatGPT Help Novice Programmers Write Better Code? Results From Static Code Analysis. https://app.cochat.ai/papers/10.1109/access.2024.3445432
The paper
Haindl, P., & Weinberger, G. (2024). Does ChatGPT Help Novice Programmers Write Better Code? Results From Static Code Analysis. IEEE Access. https://doi.org/10.1109/access.2024.3445432

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.