CoChat
Sign Up
Cited by 77Teaching and Learning ProgrammingArtificial Intelligence in EducationSoftware Engineering Research

CS1-LLM: Integrating LLMs into CS1 Instruction

Open access
Vadaparty, Annapurna, Zingaro, Daniel, Smith IV, David H., Padala, Mounika, Alvarado, Christine, Gorson Benario, Jamie, Porter, Leo·2024·Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1
DOI 10.1145/3649217.3653584
Open this paper in CoChat — free account

Abstract

The recent, widespread availability of Large Language Models (LLMs) like ChatGPT and GitHub Copilot may impact introductory programming courses (CS1) both in terms of what should be taught and how to teach it. Indeed, recent research has shown that LLMs are capable of solving the majority of the assignments and exams we previously used in CS1. In addition, professional software engineers are often using these tools, raising the question of whether we should be training our students in their use as well. This experience report describes a CS1 course at a large research-intensive university that fully embraces the use of LLMs from the beginning of the course. To incorporate the LLMs, the course was intentionally altered to reduce emphasis on syntax and writing code from scratch. Instead, the course now emphasizes skills needed to successfully produce software with an LLM. This includes explaining code, testing code, and decomposing large problems into small functions that are solvable by an LLM. In addition to frequent, formative assessments of these skills, students were given three large, open-ended projects in three separate domains (data science, image processing, and game design) that allowed them to showcase their creativity in topics of their choosing. In an end-of-term survey, students reported that they appreciated learning with the assistance of the LLM and that they interacted with the LLM in a variety of ways when writing code. We provide lessons learned for instructors who may wish to incorporate LLMs into their course.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3649217.3653584

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 16

  1. [1][1] Joe Michael Allen, Frank Vahid, Alex Edgcomb, Kelly Downey, and Kris Miller. 2019. An analysis of using many small programs in CS1. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education . 585-591. -
  2. [2][2] Michelle Craig, Phill Conrad, Dylan Lynch, Natasha Lee, and Laura Anthony. 2018. Listening to early career software developers. Journal of Computing Sciences in Colleges 33, 4 (2018), 138-149. -
  3. [4][4] Paul Denny, Andrew Luxton-Reilly, and Beth Simon. 2008. Evaluating a new exam question: Parsons problems. In Proceedings of the 4th International Workshop on Computing Education Research . 113-124. -
  4. [3][3] Catherine H Crouch and Eric Mazur. 2001. Peer instruction: Ten years of experience and results. American Journal of Physics 69, 9 (2001), 970-977. -
  5. [5][5] Mark Guzdial. 2003. A media computation course for non-majors. In Proceedings of the 8th Annual Conference on Innovation and Technology in Computer Science Education . 104-108. -
  6. [6][6] Carol Hulls, Chris Rennick, Sanjeev Bedi, Mary Robinson, and William Melek. 2015. The use of an open-ended project to improve the student experience in first year programming. Proceedings of the Canadian Engineering Education Association (CEEA) (2015). -
  7. [10][10] Charlie McDowell, Linda Werner, Heather E Bullock, and Julian Fernald. 2006. Pair programming improves student retention, confidence, and program quality. Commun. ACM 49, 8 (2006), 90-95. -
  8. [9][9] Sam Lau and Philip Guo. 2023. From "Ban It Till We Understand It" to "Resistance is Futile": How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools Such as ChatGPT and GitHub Copilot. In Proceedings of the 19th ACM Conference on International Computing Education Research . 106-121. -
  9. [11][11] Leo Porter, Dennis Bouvier, Quintin Cutts, Scott Grissom, Cynthia Lee, Robert McCartney, Daniel Zingaro, and Beth Simon. 2016. A multi-institutional study of peer instruction in introductory computing. In Proceedings of the 47th ACM Technical Symposium on Computer Science Education . 358-363. -
  10. [15][15] Adrian Salguero, Julian McAuley, Beth Simon, and Leo Porter. 2020. A longitudinal evaluation of a best practices CS1. In Proceedings of the 16th ACM Conference on International Computing Education Research . 182-193. -
  11. [18][18] Sadia Sharmin, Daniel Zingaro, and Clare Brett. 2020. Weekly Open-Ended Exercises and Student Motivation in CS1. In Proceedings of the 20th Koli Calling International Conference on Computing Education Research . -
  12. [17][17] Sadia Sharmin. 2021. Creativity in CS1: A Literature Review. ACMTrans. Comput. Educ. 22, 2 (2021). -
  13. [19][19] David H Smith IV and Craig Zilles. 2023. Code Generation Based Grading: Evaluating an Auto-grading Mechanism for 'Explain-in-Plain-English' Questions. arXiv preprint arXiv:2311.14903 (2023). - · arXiv
  14. [21][21] Anne Venables, Grace Tan, and Raymond Lister. 2009. A closer look at tracing, explaining and code writing skills in the novice programmer. In Proceedings of the 5th International Workshop on Computing Education Research . 117-128. -
  15. [20][20] Sander Valstar, Caroline Sih, Sophia Krause-Levy, Leo Porter, and William G. Griswold. 2020. A Quantitative Study of Faculty Views on the Goals of an Undergraduate CS Program and Preparing Students for Industry. In Proceedings of the 17th ACM Conference on International Computing Education Research . 113123. -
  16. [22][22] Matthew West, Geoffrey L Herman, and Craig Zilles. 2015. PrairieLearn: Masterybased online problem solving with adaptive scoring and recommendations driven by machine learning. In 2015 ASEE Annual Conference & Exposition . 26-1238. -

Cited by

Sorted by citations

Authors
Vadaparty, A., Zingaro, D., Smith IV, D. H., Padala, M., Alvarado, C., Gorson Benario, J., & Porter, L.
Affiliations
  • University of California San Diego (US)
  • University of Toronto (CA)
  • University of Illinois, Urbana, IL, USA
  • Google, Chicago, IL, USA
ORCID
Venue
Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1
Publisher
ACM
Published
2024-07-03
Pages
297-303
DOI
10.1145/3649217.3653584
OpenAlex
W4400267690
Cited by
77
References
2
Citations by year
2026: 28 · 2025: 45 · 2024: 4
FWCI
33.77
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) (publishedVersion, cc-by-nd)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Google - Award for Inclusion Research
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
large language models, introductory programming, ChatGPT, GitHub Copilot, code explanation, code testing, problem decomposition, open-ended projects, student perceptions, LLM-assisted programming

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review