The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Identifying the Factors That Influence Trust in AI Code Completion
Abstract
AI-powered software development tooling is changing the way that developers interact with tools and write code. However, the ability for AI to truly transform software development may depend on developers' levels of trust in these tools, which has consequences for tool adoption and repeated usage. In this work, we take a mixed-methods approach to measure the factors that influence developers' trust in AI-powered code completion. We found that characteristics about the AI suggestion itself (e.g., the quality of the suggestion), the developer interacting with the suggestion (e.g., their expertise in a language), and the context of the development work (e.g., was the suggestion in a test file) all influenced acceptance rates of AI-powered code suggestions. Based on these findings we propose a number of recommendations for the design of AI-powered development tools to improve trust.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Brown et al. [18] found that developers' trust in AI-code generators depends on perceived accuracy, expertise, and the risk of accepting incorrect suggestions.”
From What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI (2025) · cites this paper as [18]
Citations
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Thematic analysis of qualitative data: AMEE Guide No. 131
Kiger, M. E., & Varpio, L. (2020) · Medical Teacher · doi:10.1080/0142159x.2020.1755030 · Cited by 3,787
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“Too much trust when it is not warranted is risky [9, 15].”
BackgroundInternational Journal of Human-Computer Studies
Sdol, E. (2009) · Cited by 1,095
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
O'Donovan, J., & Smyth, B. (2005) · Proceedings of the 10th international conference on Intelligent user interfaces · doi:10.1145/1040830.1040870 · Cited by 865
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
How We Refactor, and How We Know It
Murphy-Hill, E., Parnin, C., & Black, A. P. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.41 · Cited by 547
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 2×
“Prior work has leveraged various behavioral indicators in order to measure trust in the output from ML-and AI-based systems [14, 26, 27].”
Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions
Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., & Karri, R. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833571 · Cited by 476
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436 · cited 2×
“Many early results appear to support these hypotheses, either in terms of objective measurements (e.g., time to complete a development task) or self-reported productivity [5, 7, 25, 28].”
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · cited 2×
“Prior work has leveraged various behavioral indicators in order to measure trust in the output from ML-and AI-based systems [14, 26, 27].”
Automated Unit Test Improvement using Large Language Models at Meta
Alshahwan, N., Chheda, J., Finogenova, A., Gokkaya, B. U., Harman, M., Harper, I., Marginean, A., Sengupta, S., & Wang, E. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663839 · Cited by 316
Do Users Write More Insecure Code with AI Assistants?
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3623157 · Cited by 259
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534864 · Cited by 200
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Simister, S., Sittampalam, G., Li, A., Rice, A., Rifkin, D., & Aftandilian, E. (2022) · arXiv · doi:10.48550/arxiv.2205.06537 · Cited by 200
Trust in Generative AI among Students: An exploratory study
Amoozadeh, M., Daniels, D., Nam, D., Kumar, A., Chen, S., Hilton, M., Srinivasa Ragavan, S., & Alipour, M. A. (2024) · Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3626252.3630842 · Cited by 143
Trust in Generative AI among students: An Exploratory Study
Amoozadeh, M., Daniels, D., Nam, D., Kumar, A., Chen, S., Hilton, M., Ragavan, S. S., & Alipour, M. A. (2023) · arXiv · doi:10.48550/arxiv.2310.04631 · Cited by 143
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2022) · arXiv · doi:10.48550/arxiv.2210.14306 · Cited by 114 · cited 2×
“AI-powered software development tooling is transforming the way that developers write code [4, 17].”
BackgroundTake It, Leave It, or Fix It: Measuring Productivity and Trust in Human-AI Collaboration
Qian, C., & Wexler, J. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645198 · Cited by 62
Make Your Tools Sparkle with Trust: The PICSE Framework for Trust in Software Tools
Johnson, B., Bird, C., Ford, D., Forsgren, N., & Zimmermann, T. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip58684.2023.00043 · Cited by 27
Multi-line AI-Assisted Code Authoring
Dunay, O., Cheng, D., Tait, A., Thakkar, P., Rigby, P. C., Chiu, A., Ahmad, I., Ganesan, A., Maddila, C., Murali, V., et al. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663836 · Cited by 24
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2022) · arXiv · doi:10.48550/arxiv.2206.15000 · Cited by 23 · cited 2×
“Many early results appear to support these hypotheses, either in terms of objective measurements (e.g., time to complete a development task) or self-reported productivity [5, 7, 25, 28].”
AI in Software Engineering at Facebook
Bader, J., Seohyun Kim, S., Sifei Luan, F., Chandra, S., & Meijer, E. (2021) · IEEE Software · doi:10.1109/ms.2021.3061664 · Cited by 16
Do Users Write More Insecure Code with AI Assistants?
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2022) · arXiv · cited 1×
“For example, prior work has shown that developers can write less secure code when they use AI-powered coding tools [21, 22].”
Background
Research works printed but not linked: 2
- 10.1214/aos · doi:10.1214/aos
- Hussein Mozannar Gagan Bansal Adam Fourney and Eric Horvitz. 2023. When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming. arxiv:cs.HC/2306.04930.
Cited by
How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering
Treude, C., & Gerosa, M. A. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00033 · Cited by 58
How much does AI impact development speed? An enterprise-based randomized controlled trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2024) · arXiv · doi:10.48550/arxiv.2410.12944 · Cited by 20
How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00060 · Cited by 20
What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI
Choudhuri, R., Trinkenreich, B., Pandita, R., Kalliamvakou, E., Steinmacher, I., Gerosa, M., Sanchez, C., & Sarma, A. (2025) · arXiv · doi:10.48550/arxiv.2505.17418 · Cited by 14
Human-AI experience in integrated development environments: a systematic literature review
Sergeyuk, A., Zakharov, I., Koshchenko, E., & Izadi, M. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10793-0 · Cited by 49
Metadata
- Authors
- Brown, A., D'Angelo, S., Murillo, A., Jaspan, C., & Green, C.
- Affiliations
- Google (United States) (US)
- Google, Twizel, New Zealand
- Google, Munich, Germany
- Venue
- Proceedings of the 1st ACM International Conference on AI-Powered Software
- Publisher
- ACM
- Published
- 2024-07-10
- Pages
- 1-9
- OpenAlex
- W4400484822
- Cited by
- 20
- References
- 21
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 1
- FWCI
- 7.61
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on AI-Powered Software (AIware) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- trust in AI, developer trust, tool adoption, developer expertise, test files
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.