The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
On the Use of ChatGPT for Code Review: Do Developers Like Reviews By ChatGPT?
Abstract
Code review is a critical but time-consuming process for ensuring code quality in modern software engineering. To alleviate the effort of reviewing source code, recent studies have investigated the possibility of automating the review process. Moreover, tools based on large language models such as ChatGPT are playing an increasingly important role in this vision. Understanding how these tools are used during code review can provide valuable insights for code review automation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“They found that 30.7% of AI-generated suggestions during code reviews were met with skepticism or disagreement, and most of them were about the generated code.”
From On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub (2026) · cites this paper as Watanabe et al. [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Expectations, outcomes, and challenges of modern code review
Bacchelli, A., & Bird, C. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606617 · Cited by 650
McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597076 · Cited by 348
An Analysis of the Automatic Bug Fixing Performance of ChatGPT
Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023) · 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr59189.2023.00012 · Cited by 338
An Exploratory Study on Self-Admitted Technical Debt
Potdar, A., & Shihab, E. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.31 · Cited by 281
Modern code review: a case study at google
Sadowski, C., Söderberg, E., Church, L., Sipko, M., & Bacchelli, A. (2018) · Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice · doi:10.1145/3183519.3183525 · Cited by 280
On Learning Meaningful Code Changes Via Neural Machine Translation
Tufano, M., Pantiuchina, J., Watson, C., Bavota, G., & Poshyvanyk, D. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00021 · Cited by 207
Automating code review activities by large-scale pre-training
Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., et al. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549081 · Cited by 196
Towards Automating Code Review Activities
Tufano, R., Pascarella, L., Tufano, M., Poshyvanyk, D., & Bavota, G. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00027 · Cited by 188
Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study
Guo, Q., Cao, J., Xie, X., Liu, S., Li, X., Chen, B., & Peng, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623306 · Cited by 186
Research works printed but not linked: 5
- 10.5555/3155562.3155579 · doi:10.5555/3155562.3155579
- Eman Abdullah AlOmar, Anushkrishna Venkatakrishnan, Mohamed Wiem Mkaouer, Christian D. Newman, and Ali Ouni. 2024. How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations. CoRR abs/2402.06013 (2024). · arXiv
- Carolyn D. Egelman, Emerson R. Murphy-Hill, Elizabeth Kammer, Margaret Morrow Hodges, Collin Green, Ciera Jaspan, and James Lin. 2020. Predicting developers’ negative feelings about code review. In Proc. of the 42nd International Conference on Software Engineering (ICSE’20). 174–185.
- Samia Kabir, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2023. Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions. CoRR abs/2308.02312 (2023). · arXiv
- Peter C. Rigby and Christian Bird. 2013. Convergent contemporary software peer review practices. In Proc. of the 2013 Joint Meeting of the European Software Engineering Conference (FSE’13). 202–212.
Cited by
On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
Watanabe, M., Li, H., Kashiwa, Y., Reid, B., Iida, H., & Hassan, A. E. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3798166 · Cited by 34
Self-citation
Metadata
- Authors
- Watanabe, M., Kashiwa, Y., Lin, B., Hirao, T., Yamaguchi, K., & Iida, H.
- Affiliations
- Nara Institute of Science and Technology, Japan
- Radboud University, Netherlands
- Nara College, National Institute of Technology, Japan
- Venue
- Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering
- Publisher
- ACM
- Published
- 2024-06-18
- Pages
- 375-380
- Cited by
- 28
- References
- 28
- Funders
- Japan Science and Technology Agency (JPMJPR22P3)
- Japan Society for the Promotion of Science (P21H03416, JP24K02921, JPJSBP120239929)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the Use of ChatGPT for Code Review: Do Developers Like Reviews By ChatGPT? https://app.cochat.ai/papers/10.1145/3661167.3661183
- The paper
- Watanabe, M., Kashiwa, Y., Lin, B., Hirao, T., Yamaguchi, K., & Iida, H. (2024). On the Use of ChatGPT for Code Review: Do Developers Like Reviews By ChatGPT? Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering. https://doi.org/10.1145/3661167.3661183
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.