The DOI matches its Crossref record; 25 of the 26 references it cites were found in a registry or an index, and 1 are not checked yet.
Students' Use of GitHub Copilot for Working with Large Code Bases
Open accessAbstract
Large language models (LLMs) are already heavily used by professional software engineers. An important skill for new university graduates to possess will be the ability to use such LLMs to effectively navigate and modify a large code base. While much of the prior work related to LLMs in computing education focuses on novice programmers learning to code, less work has focused on how upper-division students use and trust these tools, especially while working with large code bases. In this study, we taught students about various GitHub Copilot features, including Copilot chat, in an upper-division software engineering course and asked students to add a feature to a large code base using Copilot. Our analysis revealed a novel interaction pattern that we call one-shot prompting, in which students ask Copilot to implement the entire feature at once and spend the next few prompts asking Copilot to debug the code or asking Copilot to regenerate its incorrect response. Finally, students reported significantly more trust in the code comprehension features than code generation features of Copilot, perhaps due to the presence of trust affordances in the Copilot chat that are absent in the code generation features. Our study takes the first steps in understanding how upper-division students use Github Copilot so that our instruction can adequately prepare students for a career in software engineering.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Generative Artificial Intelligence (AI) tools such as ChatGPT, GitHub Copilot, and Gemini have become embedded in the daily workflow of graduate students [12, 17, 19].”
From Students' Practices and Skills in the LLM-Era: "You Can't Outsource the Struggle and Still Get the Skill" (2026) · cites this paper as [19]
In the citing paper
“Furthermore, our finding that Copilot reduced manual coding comports with Shah et. al.'s [36] observation that students perform 'one-shot prompting' to have Copilot implement entire features at once.”
From The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Coding Tasks (2025) · cites this paper as [36]
Citations
Cites
The Discovery of Grounded Theory: Strategies for Qualitative Research
Glaser, B. G., & Strauss, A. L. (2017) · Aldine (Routledge reprint) · doi:10.4324/9780203793206 · Cited by 25,789
Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.116 · Cited by 685
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J. (2022) · Proceedings of the 24th Australasian Computing Education Conference · doi:10.1145/3511861.3511863 · Cited by 463
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436
Do Users Write More Insecure Code with AI Assistants?
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3623157 · Cited by 259
Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256
Ozkaya, I. (2023) · IEEE Software · doi:10.1109/ms.2023.3248401 · Cited by 206
Investigating the skill gap between graduating students and industry expectations
Radermacher, A., Walia, G., & Knudson, D. (2014) · Companion Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2591062.2591159 · Cited by 199
Struggles of new college graduates in their first software development job
Begel, A., & Simon, B. (2008) · Proceedings of the 39th SIGCSE technical symposium on Computer science education · doi:10.1145/1352135.1352218 · Cited by 193
Research works printed but not linked: 3
- Mateusz Jaworski and Dariusz Piotrkowski. 2023. Study of software developers' experience using the Github Copilot Tool in the software development process. ArXiv, Vol. abs/2301.04991 (2023). https://api.semanticscholar.org/CorpusID:255749277 · arXiv
- Sida Peng Eirini Kalliamvakou Peter Cihon and Mert Demirer. 2023. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arxiv: 2302.06590 · arXiv
- 10.1007/978--94--6351-086--8_4 · doi:10.1007/978--94--6351-086--8_4
Cited by
Exploring Student-AI Interactions in Vibe Coding
Geng, F., Shah, A., Li, H., Mulla, N., Swanson, S., Raj, G. S., Zingaro, D., & Porter, L. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786236 · Cited by 43
Self-citationBernstein, S., Rahman, A., Sharifi, N., Terbish, A., & MacNeil, S. (2025) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3769994.3770036 · Cited by 37
Shihab, M. I. H., Hundhausen, C., Tariq, A., Haque, S., Qiao, Y., & Mulanda, B. W. (2025) · Proceedings of the 2025 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3702652.3744219 · Cited by 28
Kharrufa, A., Alghamdi, S. S., Aziz, A., & Bull, C. (2025) · ACM Transactions on Computing Education · doi:10.1145/3779296 · Cited by 36
"My productivity is boosted, but …" Demystifying Users’ Perception on AI Coding Assistants
Lyu, Y., Yang, Z., Shi, J., Chang, J., Liu, Y., & Lo, D. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00024 · Cited by 19
Metadata
- Authors
- Shah, A., Chernova, A., Tomson, E., Porter, L., Griswold, W. G., & Soosai Raj, A. G.
- Affiliations
- University of California San Diego (US)
- University of California, San Diego, La Jolla, CA, USA
- ORCID
- Venue
- Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1
- Publisher
- ACM
- Published
- 2025-02-12
- Pages
- 1050-1056
- OpenAlex
- W4407771444
- Cited by
- 19
- References
- 21
- Citations by year
- 2026: 12 · 2025: 7
- FWCI
- 27.61
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion, cc-by)
- Funders
- NSF (National Science Foundation) (2417531)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- GitHub Copilot, large language models, large codebases, one-shot prompting, program comprehension, code generation, feature implementation, debugging, software engineering education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Students' Use of GitHub Copilot for Working with Large Code Bases. https://app.cochat.ai/papers/10.1145/3641554.3701800
- The paper
- Shah, A., Chernova, A., Tomson, E., Porter, L., Griswold, W. G., & Soosai Raj, A. G. (2025). Students' Use of GitHub Copilot for Working with Large Code Bases. Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1. https://doi.org/10.1145/3641554.3701800
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.