A Review of Peer Code Review in Higher Education
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Peer review is the standard process within academia for maintaining publication quality, but it is also widely employed in other settings, such as education and industry, for improving work quality and for generating actionable feedback to content authors. For example, in the software industry peer review of program source code—or peer code review—is a key technique for detecting bugs and maintaining coding standards. In a programming education context, although peer code review offers potential benefits to both code reviewers and code authors, individuals are typically less experienced, which presents a number of challenges. Some of these challenges are similar to those reported in the educational literature on peer review in other academic disciplines, but reviewing code presents unique difficulties. Better understanding these challenges and the conditions under which code review can be taught and implemented successfully in computer science courses is of value to the computing education community. In this work, we conduct a systematic review of the literature on peer code review in higher education to examine instructor motivations for conducting peer code review activities, how such activities have been implemented in practice, and the primary benefits and difficulties that have been reported. We initially identified 187 potential studies and analyzed 51 empirical studies pertinent to our goals. We report the most commonly cited benefits (e.g., the development of programming-related skills) and barriers (e.g., low student engagement), and we identify a wide variety of tools that have been used to facilitate the peer code review process. While we argue that more empirical work is needed to validate currently reported results related to learning outcomes, there is also a clear need to address the challenges around student motivation, which we believe could be an important avenue for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 1×
“Snowballing refers to the identification of additional papers using the reference list of an article or the citations to the article [83].”
BackgroundGamification. using game-design elements in non-gaming contexts
Deterding, S., Sicart, M., Nacke, L., O'Hara, K., & Dixon, D. (2011) · CHI '11 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1979742.1979575 · Cited by 2,413
Peer Assessment between Students in Colleges and Universities
Topping, K. (1998) · Review of Educational Research · doi:10.2307/1170598 · Cited by 2,173 · cited 3×
“Benefits to students include the development of social and communication skills as well as active learning through discussion and collaboration with their peers around course related content [15, 70].”
BackgroundRethinking Assessment in Higher Education: Learning for the Longer Term
(2007) · doi:10.4324/9780203964309 · Cited by 1,049
Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1545896 · Cited by 306
Panadero, E., & Alqassab, M. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2019.1600186 · Cited by 232 · cited 1×
“The implementation of student peer review in higher education is growing [16, 25, 50].”
BackgroundRamon-Casas, M., Nuño, N., Pons, F., & Cunillera, T. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1525337 · Cited by 27
Cited by
The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J. (2022) · Proceedings of the 24th Australasian Computing Education Conference · doi:10.1145/3511861.3511863 · Cited by 463
“The extensive literature on peer review, including the review of code [17, 21] makes it clear that there are many benefits that arise from looking at a variety of solutions to a given problem.”
BackgroundSelf-citationBecker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569759 · Cited by 401
Self-citationComputing Education in the Era of Generative AI
Denny, P., Prather, J., Becker, B. A., Finnie-Ansley, J., Hellas, A., Leinonen, J., Luxton-Reilly, A., Reeves, B. N., Santos, E. A., & Sarsa, S. (2024) · Communications of the ACM · doi:10.1145/3624720 · Cited by 316
Self-citationBecker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2022) · arXiv · doi:10.48550/arxiv.2212.01020 · Cited by 36
Self-citationNeshaei, S. P., Rietsche, R., Su, X., & Wambsganss, T. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645169 · Cited by 9
“Peer review is also used in the code-reviewing process of software development to detect bugs and maintain coding standards [50].”
Method
Metadata
- Authors
- Indriasari, T. D., Luxton-Reilly, A., & Denny, P.
- Venue
- ACM Transactions on Computing Education
- Published
- 2020-09-09
- Volume
- 20
- Issue
- 3
- Pages
- 1-25
- ISSN
- 1946-6226 (ISSN-L)
- OpenAlex
- W3086641389
- Cited by
- 80
- References
- 7
- FWCI
- 7.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Review of Peer Code Review in Higher Education. https://app.cochat.ai/papers/10.1145/3403935
- The paper
- Indriasari, T. D., Luxton-Reilly, A., & Denny, P. (2020). A Review of Peer Code Review in Higher Education. ACM Transactions on Computing Education. https://doi.org/10.1145/3403935
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.