Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index; 17 of the 49 sources its claims cite were checked for support.

My Code Is Less Secure with Gen AI: Surveying Developers' Perceptions of the Impact of Code Generation Tools on Security

Open access
Kudriavtseva, Arina, Hotak, Nisar Ahmad, Gadyatskaya, Olga · 2025 · Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing

Abstract

Background: Generative AI (GAI) tools like GitHub Copilot and ChatGPT are transforming software development by automating code generation and enhancing developers' productivity. However, since these tools are often trained on open-source repositories, they may inadvertently reproduce vulnerable code, raising concerns about the security of AI-generated outputs. Aims: In this paper, we aim to investigate how developers perceive code security when using GAI tools. Method: We conducted a survey with 105 software developers with diverse experience levels to gather their perceptions regarding the security of generated code and their suggestions for improving it. Results: While developers reported increased development speed when using GAI tools, many spend additional time on security reviews and documentation of the generated code, and they are worried about the overreliance on AI and vulnerabilities in the code. Only about a quarter of the developers expressed confidence in the code generated by AI, and, moreover, experienced developers perceive that their proficiency in secure coding decreases when using GAI tools. Our results provide organizations with a better understanding of the risks associated with GAI tools and help improve their software security programs.

Overview

CoChat Summary

## TL;DR

This survey study of 105 developers examines how people perceive the security impact of generative AI code tools like ChatGPT and GitHub Copilot. Developers reported productivity gains, but many also reported more security review, low confidence in AI-generated code, and concerns about overreliance and vulnerabilities.

## Key findings

  • Among 105 valid respondents, 78% named faster development as a benefit and 65% cited increased efficiency; 36% also saw improved vulnerability detection and 32% improved maintainability.
  • 61% said they spend less time writing code with GAI tools, but 44% said they spend more time on security reviews, and 49% said documenting generated code is more important.
  • Perceived risks were led by overreliance on AI without human oversight (65%), code with similar vulnerabilities (59%), introduction of new vulnerabilities (52%), and privacy/ethical concerns (48%).
  • Only 25% were confident in the security of AI-generated code, while 43% lacked confidence; 89% reported using ChatGPT and 59% GitHub Copilot, and 62% of IntelliCode users were confident in its security.

## Methods

  • The authors ran an anonymous survey using Qualtrics from May 19 to July 13, 2024, targeting software developers; the study was approved by Leiden University’s Science Ethics Committee.
  • They received 196 responses and kept 105 valid responses; participants were grouped by experience into junior (student/learning, 0–2 years), medior (3–5 years), and senior (more than 6 years).
  • Analysis combined descriptive statistics, open-ended theme extraction, and statistical tests: two Kruskal-Wallis tests and one Brunner-Munzel test, with Bonferroni correction at p = 0.016.

## Limitations

  • The paper states the sample may have self-selection bias and non-response bias, and that the findings may not generalize to the whole developer population because the sample size was 105.
  • The survey format limited follow-up questions and multiselect options, which may have constrained depth; this is stated in the paper.
  • The study relied on self-reported answers, so misreporting or misunderstanding is possible; this is stated in the paper.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3672608.3707778

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Demographic information of the participants.
Figure 1 Demographic information of the participants.From the PDF
Figure 2 Change in the proficiency level in writing secure code with GAI tools.
Figure 2 Change in the proficiency level in writing secure code with GAI tools.From the PDF
Figure 3 Agreement with statements ' I spend less time writing code with GAI ' and ' I spend more time reviewing generated code ': comparison among experience levels.
Figure 3 Agreement with statements ' I spend less time writing code with GAI ' and ' I spend more time reviewing generated code ': comparison among experience levels.From the PDF
Figure 4 Perceived benefits for code security of using generative AI tools, per experience level.
Figure 4 Perceived benefits for code security of using generative AI tools, per experience level.From the PDF
Figure 5 The perceived risks of using generative AI in code security, per experience level.
Figure 5 The perceived risks of using generative AI in code security, per experience level.From the PDF
Figure 6 Security practices recommended by developers.
Figure 6 Security practices recommended by developers.From the PDF
Figure 7 Boxplots of the rating of factors that contribute to code security according to the participants.
Figure 7 Boxplots of the rating of factors that contribute to code security according to the participants.From the PDF
Figure 8 Agreement with the statement ' There is a higher risk of security vulnerabilities in code generated by full program creation GAI tools compared to code completion tools '.
Figure 8 Agreement with the statement ' There is a higher risk of security vulnerabilities in code generated by full program creation GAI tools compared to code completion tools '.From the PDF
Figure 9 Answers to the question ' How confident are you in the security of code generated by AI tools compared to code written by yourself? '.
Figure 9 Answers to the question ' How confident are you in the security of code generated by AI tools compared to code written by yourself? '.From the PDF

Verification

Claims the paper makes

14 Valid · 2 Not valid · 15 Not checked · 100 Own results and interpretations

41 more claims

Create a free account to see all 131 claims and to verify the 15 references still open.

See all claims — free account
  • Probably not a substantial claim. 49
  • It may state two separate claims as one. 46
  • The quote only partly states this claim. 4

Create a free account to see these claims.

Citations

Cites

Show all 25 references

Research works printed but not linked: 1

  1. Johan Linåker, Sardar Muhammad Sulaman, Rafael Maiani de Mello, and Martin Höst. 2015. Guidelines for conducting surveys in software engineering. (2015).

Cited by

See all 9

Metadata

Authors
Kudriavtseva, A., Hotak, N. A., & Gadyatskaya, O.
Affiliations
Leiden University (NL)
Venue
Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing
Publisher
ACM
Published
2025-03-31
Pages
1637-1646
OpenAlex
W4410357414
Cited by
9
References
25
Citations by year
2026: 9
FWCI
10.74
Type
conference paper
Language
en
Open access
Locations
  • ACM Symposium on Applied Computing (publishedVersion, cc-by-nc-sa)
  • Leiden Repository (Leiden University) (submittedVersion, cc-by-sa)
Funders
NWO (NWA.1215.18.008)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
generative AI, code generation tools, GitHub Copilot, ChatGPT, software security, developer perceptions, secure coding, security review, developer experience

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: My Code Is Less Secure with Gen AI: Surveying Developers' Perceptions of the Impact of Code Generation Tools on Security. https://app.cochat.ai/papers/10.1145/3672608.3707778
The paper
Kudriavtseva, A., Hotak, N. A., & Gadyatskaya, O. (2025). My Code Is Less Secure with Gen AI: Surveying Developers' Perceptions of the Impact of Code Generation Tools on Security. Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing. https://doi.org/10.1145/3672608.3707778

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.