Code Review Automation: Strengths and Weaknesses of the State of the Art
Open accessAbstract
The automation of code review has been tackled by several researchers with the goal of reducing its cost. The adoption of deep learning in software engineering pushed the automation to new boundaries, with techniquesimitatingdevelopers in generative tasks, such as commenting on a code change as a reviewer would do or addressing a reviewer's comment by modifying code. The performance of these techniques is usually assessed through quantitative metrics,e.g.,the percentage of instances in the test set for which correct predictions are generated, leaving many open questions on the techniques’ capabilities. For example, knowing that an approach is able to correctly address a reviewer's comment in 10% of cases is of little value without knowing what was asked by the reviewer: What if in all successful cases the code change required to address the comment was just the removal of an empty line? In this paper we aim at characterizing the cases in which three code review automation techniques tend to succeed or fail in the two above-described tasks. The study has a strong qualitative focus, with$\sim$105 man-hours of manual inspection invested in manually analyzing correct and wrong predictions generated by the three techniques, for a total of 2,291 inspected predictions. The output of this analysis are two taxonomies reporting, for each of the two tasks, the types of code changes on which the experimented techniques tend to succeed or to fail, pointing to areas for future work. A result of our manual analysis was also the identification of several issues in the datasets used to train and test the experimented techniques. Finally, we assess the importance of researching in techniques specialized for code review automation by comparing their performance with ChatGPT, a general purpose large language model, finding that ChatGPT struggles in commenting code as a human reviewer would do.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent assessments of automated code review report mixed results, noting that ChatGPT lacks skill in understanding the larger context of a program, as opposed to a small bit of code, as well as with making recommendations and implementing changes.
“Recent assessments of the accuracy of automated code review report quite mixed results, noting that ChatGPT lacks skill in understanding the larger context of a program, as opposed to a small bit of a code, as well as with making recommendations and implementing changes (Tufano et al. 2024).”
From Large language models and the problem of rhetorical debt (2025) · cites this paper as Tufano et al. 2024
Cites
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · see citing works
EvoSuite: automatic test suite generation for object-oriented software
Fraser, G., & Arcuri, A. (2011) · ESEC/FSE '11 · doi:10.1145/2025113.2025179 · Cited by 1,445 · see citing works
code2vec: learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 1,235 · see citing works
Effect sizes for research: A broad practical approach.
Grissom, R. J., & Kim, J. J. (2005) · Cited by 732 · see citing works
Expectations, outcomes, and challenges of modern code review
Bacchelli, A., & Bird, C. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606617 · Cited by 650 · see citing works
Fine-grained and accurate source code differencing
Falleri, J., Morandat, F., Blanc, X., Martínez, M., & Monperrus, M. (2014) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2642937.2642982 · Cited by 569 · see citing works
Rao, K. J. P. H., Janardhan, K., & Rao, P. (2010) · Cited by 559 · see citing works
Pacheco, C., & Ernst, M. D. (2007) · Conference on Object-Oriented Programming Systems, Languages, and Applications · doi:10.1145/1297846.1297902 · Cited by 558 · see citing works
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D. (2019) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3340544 · Cited by 370 · see citing works
Self-citationMcIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597076 · Cited by 351 · see citing works
Convergent contemporary software peer review practices
Rigby, P. C., & Bird, C. (2013) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2491411.2491444 · Cited by 315 · see citing works
Sadowski, C., Söderberg, E., Church, L., Sipko, M., & Bacchelli, A. (2018) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3183519.3183525 · Cited by 280 · see citing works
(2014) · doi:10.1145/2597073.2597082 · Cited by 252 · see citing works
Yu, Y., Wang, H., Yin, G., & Wang, T. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.01.004 · Cited by 240 · see citing works
Thongtanunam, P., Tantithamthavorn, C. K., Kula, R. G., Yoshida, N., Iida, H., & Matsumoto, K. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081824 · Cited by 237 · see citing works
Towards Automating Code Review Activities
Tufano, R., Pascarella, L., Tufano, M., Poshyvanyk, D., & Bavota, G. (2021) · International Conference on Software Engineering · doi:10.1109/icse43902.2021.00027 · Cited by 230 · see citing works
Self-citationAutomating code review activities by large-scale pre-training
Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., et al. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549081 · Cited by 226 · see citing works
On Learning Meaningful Code Changes Via Neural Machine Translation
Tufano, M., Pantiuchina, J., Watson, C., Bavota, G., & Poshyvanyk, D. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00021 · Cited by 212 · see citing works
Self-citationTsantalis, N., Ketkar, A., & Dig, D. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3007722 · Cited by 212 · see citing works
What Types of Defects Are Really Discovered in Code Reviews?
Mäntylä, M., & Lassenius, C. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.71 · Cited by 203 · see citing works
Using pre-trained models to boost code review automation
Tufano, R., Masiero, S., Mastropaolo, A., Pascarella, L., Poshyvanyk, D., & Bavota, G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510621 · Cited by 182 · see citing works
Self-citationAutomatically Recommending Peer Reviewers in Modern Code Review
Zanjani, M. B., Kagdi, H., & Bird, C. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2015.2500238 · Cited by 175 · see citing works
On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot
Mastropaolo, A., Pascarella, L., Guglielmi, E., Ciniselli, M., Scalabrino, S., Oliveto, R., & Bavota, G. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00181 · Cited by 149 · see citing works
Self-citationDo code review practices impact design quality? A case study of the Qt, VTK, and ITK projects
Morales, R., McIntosh, S., & Khomh, F. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081827 · Cited by 140 · see citing works
(2018) · doi:10.1145/3274404 · Cited by 122 · see citing works
Peer Review on Open-Source Software Projects
Rigby, P. C., Germán, D. M., Cowen, L. L. E., & Storey, M. (2014) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2594458 · Cited by 110 · see citing works
CoRReCT: code reviewer recommendation in GitHub based on cross-project and technology experience
Rahman, M. M., Roy, C. K., & Collins, J. A. (2016) · Proceedings of the 38th International Conference on Software Engineering Companion · doi:10.1145/2889160.2889244 · Cited by 109 · see citing works
Xia, X., Lo, D., Wang, X., & Yang, X. (2015) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2015.7332472 · Cited by 106 · see citing works
Four eyes are better than two: On the impact of code reviews on software quality
Bavota, G., & Russo, B. (2015) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332454 · Cited by 101 · see citing works
Self-citationCoxon, A. (1999) · Sage eBooks · doi:10.4135/9781412985871 · Cited by 101 · see citing works
Impact of Peer Code Review on Peer Impression Formation: A Survey
Bosu, A., & Carver, J. C. (2013) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.23 · Cited by 100 · see citing works
Search-Based Peer Reviewers Recommendation in Modern Code Review
Ouni, A., Kula, R. G., & Inoue, K. (2016) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2016.65 · Cited by 90 · see citing works
Jiang, J., Yang, Y., He, J., Blanc, X., & Zhang, L. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2016.10.006 · Cited by 76 · see citing works
Using deep learning to generate complete log statements
Mastropaolo, A., Pascarella, L., & Bavota, G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3511561 · Cited by 68 · see citing works
Self-citationAUGER: automatically generating review comments with pre-training models
Li, L., Li, Y., Jiang, H., Yan, J., Luo, T., Hua, Z., Geng, L., & Zuo, C. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549099 · Cited by 63 · see citing works
Data Augmentation by Program Transformation
Yu, S., Wang, T., & Wang, J. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2022.111304 · Cited by 61 · see citing works
Chouchen, M., Ouni, A., Mkaouer, M. W., Kula, R. G., & Inoue, K. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106908 · Cited by 60 · see citing works
Thongtanunam, P., Pornprasit, C., & Tantithamthavorn, C. K. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510067 · Cited by 59 · see citing works
CommentFinder: a simpler, faster, more accurate code review comments recommendation
Hong, Y., Tantithamthavorn, C. K., Thongtanunam, P., & Aleti, A. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549119 · Cited by 59 · see citing works
Mitigating turnover with code review recommendation
Mirsaeedi, E., & Rigby, P. C. (2020) · doi:10.1145/3377811.3380335 · Cited by 56 · see citing works
WhoDo: automating reviewer suggestions at scale
Asthana, S., Kumar, R., Bhagwan, R., Bird, C., Bansal, C., Maddila, C., Mehta, S., & Ashok, B. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3340449 · Cited by 54 · see citing works
A hybrid approach to code reviewer recommendation with collaborative filtering
Xia, Z., Sun, H., Jiang, J., Wang, X., & Liu, X. (2017) · International Workshop on Software Mining (SoftwareMining) · doi:10.1109/softwaremining.2017.8100850 · Cited by 50 · see citing works
Automatic Code Review by Learning the Revision of Source Code
Shi, S., Li, M., Lo, D., Thung, F., & Huo, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014910 · Cited by 47 · see citing works
EARec: Leveraging Expertise and Authority for Pull-Request Reviewer Recommendation in GitHub
Ying, H., Chen, L., Liang, T., & Wu, J. (2016) · 2016 IEEE/ACM 3rd International Workshop on CrowdSourcing in Software Engineering (CSI-SE) · doi:10.1145/2897659.2897660 · Cited by 44 · see citing works
Workload-aware reviewer recommendation using a multi-objective search-based approach
Al-Zubaidi, W. H. A., Thongtanunam, P., Dam, H. K., Tantithamthavorn, C. K., & Ghose, A. (2020) · doi:10.1145/3416508.3417115 · Cited by 35 · see citing works
Jiang, J., Lo, D., Zheng, J., Xia, X., Yang, Y., & Zhang, L. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.04.055 · Cited by 24 · see citing works
Adding Context to Source Code Representations for Deep Learning
Tian, F., & Treude, C. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00042 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Current Challenges of Software Engineering in the Era of Large Language Models
Gao, C., Hu, X., Gao, S., Xia, X., & Jin, Z. G. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712005 · Cited by 65 · see citing works
Automatic Programming: Large Language Models and Beyond
Lyu, M. R.-T., Ray, B., Roychoudhury, A., Tan, S. H., & Thongtanunam, P. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708519 · Cited by 65 · see citing works
A survey on large language models for software engineering
Zhang, Q., Fang, C., Xie, Y., Zhang, Y., Yu, S., Sun, W., Yang, Y., & Chen, Z. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-4670-0 · Cited by 124 · see citing works
AI-Assisted Assessment of Coding Practices in Modern Code Review
Vijayvergiya, M., Salawa, M., Budiselić, I., Zheng, D., Lamblin, P., Ivanković, M., Carin, J., Lewko, M., Andonov, J., Petrović, G., et al. (2024) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1145/3664646.3665664 · Cited by 28 · see citing works
Nashaat, M., & Miller, J. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3428324 · Cited by 26 · see citing works
Automated Code Review in Practice
Cihan, U., Haratian, V., İçöz, A., Gül, M. K., Devran, Ö., Bayendur, E. F., Uçar, B. M., & Tüzün, E. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00043 · Cited by 23 · see citing works
An Extensive Comparison of Static Application Security Testing Tools
Esposito, M., Falaschi, V., & Falessi, D. (2024) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3661167.3661199 · Cited by 23 · see citing works
Automating modern code review processes with code similarity measurement
Kartal, Y., Akdeniz, E. K., & Özkan, K. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107490 · Cited by 12 · see citing works
Intention is All you Need: Refining your Code from your Intention
Guo, Q., Xie, X., Liu, S., Hu, M., Li, X., & Bu, L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00191 · Cited by 15 · see citing works
Combining Large Language Models with Static Analyzers for Code Review Generation
Jaoua, I., Sghaier, O. B., & Sahraoui, H. A. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00038 · Cited by 13 · see citing works
Large language models and the problem of rhetorical debt
MacArthur, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-025-02403-w · Cited by 8 · see citing works
“Recent assessments of the accuracy of automated code review report quite mixed results, noting that ChatGPT lacks skill in understanding the larger context of a program, as opposed to a small bit of a code, as well as with making recommendations and implementing changes (Tufano et al. 2024).”
LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM
Zhang, Y., Zhang, Y., Sun, Z., Jiang, Y., & Liu, H. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00245 · Cited by 9 · see citing works
Sustainable software engineering—A contribution puzzle of different teams in large IT organizations
Poth, A., & Momen, P. (2024) · Journal of Software Evolution and Process · doi:10.1002/smr.2677 · Cited by 7 · see citing works
Exploring the Potential of Llama Models in Automated Code Refinement: A Replication Study
Caumartin, G., Qin, Q., Chatragadda, S., Panjrolia, J., Li, H., & Costa, D. E. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00070 · Cited by 8 · see citing works
Explaining GitHub Actions Failures with Large Language Models: Challenges, Insights, and Limitations
Valenzuela-Toledo, P., Wu, C., Hernández, S., Boll, A., Machacek, R., Panichella, S., & Kehrer, T. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00037 · Cited by 6 · see citing works
CodeDoctor: multi-category code review comment generation
Li, Y., Wu, Y., Wang, Z., Huang, L., Wang, J., Li, J. K., & Huang, M. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00491-y · Cited by 6 · see citing works
Zhang, Y., Jin, Z., Xing, Y., Li, G., Liu, F., Zhu, J., Dou, W., & Wei, J. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3718739 · Cited by 28 · see citing works
Deep Learning-based Code Reviews: A Paradigm Shift or a Double-Edged Sword?
Tufano, R., Martin-Lopez, A., Tayeb, A. J., Dabić, O., Haiduc, S., & Bavota, G. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00060 · Cited by 4 · see citing works
Self-citationLeveraging Reviewer Experience in Code Review Comment Generation
Lin, H. Y., Thongtanunam, P., Treude, C., Godfrey, M. W., Liu, C., & Charoenwet, W. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3762183 · Cited by 5 · see citing works
Hydra-Reviewer: A Holistic Multi-Agent System for Automatic Code Review Comment Generation
Ren, X., Dai, C., Huang, Q., Wang, Y., Liu, C., & Jiang, B. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3621462 · Cited by 3 · see citing works
Banitaan, S., Daoud, M., Alquran, H., & Akour, M. A. (2026) · Information · doi:10.3390/info17010073 · Cited by 5 · see citing works
Harnessing Large Language Models for Curated Code Reviews
Sghaier, O. B., Weyssow, M., & Sahraoui, H. A. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00039 · Cited by 5 · see citing works
Stress detection through prompt engineering with a general-purpose LLM
Esmi, N., Shahbahrami, A., Nabati, Y., Rezaei, B., Gaydadjiev, G. N., & de Jonge, P. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.105462 · Cited by 4 · see citing works
Too Noisy To Learn: Enhancing Data Quality for Code Review Comment Generation
Liu, C., Lin, H. Y., & Thongtanunam, P. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00043 · Cited by 3 · see citing works
Comparative Study of Code Generation by AI Models
Panthi, A., Sharma, N. K., Khanal, A., & Sharma, K. (2025) · International Conference on Computational Robotics, Testing and Engineering Evaluation (ICCRTEE) · doi:10.1109/iccrtee64519.2025.11052899 · Cited by 1 · see citing works
Liu, F., Liu, S., Zhu, Y., Lian, X., & Zhang, L. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773191 · Cited by 2 · see citing works
Beyond Static Analysis: Detecting SQL Injection via Context-Aware Code Review in Web Applications
Anggoman, C. R., Maulana, R., Setiawan, H., & Al-Farabi, M. A. N. (2025) · IEEE International Conference on Cryptography, Informatics, and Cybersecurity (ICoCICs) · doi:10.1109/icocics68032.2025.11384074 · Cited by 1 · see citing works
Cai, S. (2026) · International Conference on Electrical Automation and Artificial Intelligence (ICEAAI) · doi:10.1109/iceaai68945.2026.11442374 · Cited by 1 · see citing works
Code Reviews on a Budget: Memory-Efficient Fine-Tuning with QLoRA and RAG for Big Code Applications
Aradhya, S. N., Moh, M., & Moh, T.-S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-14107-1_31 · Cited by 1 · see citing works
A Reflection on “Advances in Software Inspections”
Porter, A., Siy, H. P., & Votta, L. G. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3537080
How AI Agents Are Transforming Software Engineering and the Future of Product Development
Panyam, S., & Gujar, P. (2025) · Computer · doi:10.1109/mc.2024.3488378
Automated Vulnerability-Focused Code Reviews
Centellas-Claros, L. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3731467
Efficode: An Intelligent System for Automated Code Review and RefactoringPreprint
Firos, Z., & Sreelakshmi, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5355169
Heterogeneous Graph Learning for Code Reviewers Recommendation with LLM's Understanding
Yao, Y., & Zheng, X. (2025) · International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA) · doi:10.1109/caibda65784.2025.11183380
Machine Learning Solutions to Automated Governance and Code Analysis to Compliance
Paul, A. A., Harshini, T. V., & Maheswari, U. (2025) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic64647.2025.11331106
What Types of Code Review Comments Do Developers Most Frequently Resolve?
Goldman, S., Lin, H. Y., Pasuksmit, J., Thongtanunam, P., Tantithamthavorn, C. K., Wang, Z., Zhang, R., Behnaz, A., Jiang, F., Siers, M. J., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00312
A Roadmap for Modern Code Review: Challenges and Opportunities
Yang, Z., Gao, C., Guo, Z., Li, Z., Liu, K., Xia, X., & Zhou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3800963
Prabakar, D., Bhardwaj, R., S, S. C., Tiwari, M., & Manikandan, G. (2026) · Journal of Computer Languages · doi:10.1016/j.cola.2026.101396
ACREF: Enhancing Code Refinement Tasks for Large Language Models Using LLM-as-a-JudgePreprint
Mi, L., Xu, L., Liu, C., Mu, W., Li, Z., & Huangfu, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6642016
Intelligent Code Analysis and Automated Testing Technologies for Power Grid Software Systems
Zhao, J. (2026) · International Conference on Power Electronics and Electric Drives (PEED) · doi:10.1109/peed69047.2026.00075
The Evolution of Code Review: From Traditional Methods to AI-Powered Approaches
İçöz, B., & Biricik, G. (2026) · Black Sea Journal of Engineering and Science · doi:10.34248/bsengineering.1635994
Toward Author-Guided Review: An Agentic Architecture for Reflective Code Review
Davila, N., & Wiese, I. (2026) · International Workshop on Agentic Engineering · doi:10.1145/3786167.3788428
Vidyavathi, K., Krishna, S., Natarajan, R., Velusamy, S., & Thangarasu, N. (2025) · International Conference on Emerging Research in Computational Science (ICERCS) · doi:10.1109/icercs65898.2025.11580587
Balancing usefulness and naturalness: an LLM-based curation pipeline for code review comments
Sghaier, O. B., Weyssow, M., & Sahraoui, H. A. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10928-x
Studying Quality Improvements Recommended via Manual and Automated Code Review
Crupi, G., Tufano, R., & Bavota, G. (2026) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3794763.3794809
Self-citationACREF: LLM-as-a-Judge Guided Data Curation and Semantic Evaluation for Automated Code RefinementPreprint
Mi, L., Xu, L., Liu, C., Mu, W., Li, Z., & Huangfu, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7346712
BCaLLM: Call Graph-Guided Python Breaking Change Detection with Large Language Models
Cheng, W., Shen, C., Zhang, H., Wu, Y., Yang, J., & Hu, W. P. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832195
Empowering Lightweight Language Models for Security Code Review via Context-Aware Distillation
Zhao, Z., Jiang, Y., Liu, H., & Zhang, L. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832293
- Authors
- Tufano, R., Dabić, O., Mastropaolo, A., Ciniselli, M., & Bavota, G.
- Affiliations
- Università della Svizzera italiana (CH)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2024-02
- Volume
- 50
- Issue
- 2
- Pages
- 338-353
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W4390480971
- Cited by
- 48
- References
- 52
- Citations by year
- 2026: 16 · 2025: 26 · 2024: 6
- FWCI
- 18.10
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Commission (HORIZON2020, 851720)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- code review automation, code comment generation, dataset quality, ChatGPT, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
