CoChat
Sign Up
PreprintCited by 15Software Engineering ResearchSoftware Engineering Techniques and PracticesTeaching and Learning Programming

Vibe Coding in Practice: Motivations, Challenges, and a Future Outlook -- a Grey Literature Review

Open access
Fawzy, Ahmed, Tahir, Amjed, Blincoe, Kelly·2025·arXiv
DOI 10.48550/arxiv.2510.00328arXiv 2510.00328
Open this paper in CoChat — free account

Abstract

AI code generation tools are transforming software development, especially for novice and non-software developers, by enabling them to write code and build applications faster and with little to no human intervention. Vibe coding is the practice where users rely on AI code generation tools through intuition and trial-and-error without necessarily understanding the underlying code. Despite widespread adoption, no research has systematically investigated why users engage in vibe coding, what they experience while doing so, and how they approach quality assurance (QA) and perceive the quality of the AI-generated code. To this end, we conduct a systematic grey literature review of 101 practitioner sources, extracting 518 firsthand behavioral accounts about vibe coding practices, challenges, and limitations. Our analysis reveals a speed-quality trade-off paradox, where vibe coders are motivated by speed and accessibility, often experiencing rapid ``instant success and flow'', yet most perceive the resulting code as fast but flawed. QA practices are frequently overlooked, with many skipping testing, relying on the models' or tools' outputs without modification, or delegating checks back to the AI code generation tools. This creates a new class of vulnerable software developers, particularly those who build a product but are unable to debug it when issues arise. We argue that vibe coding lowers barriers and accelerates prototyping, but at the cost of reliability and maintainability. These insights carry implications for tool designers and software development teams. Understanding how vibe coding is practiced today is crucial for guiding its responsible use and preventing a broader QA crisis in AI-assisted development.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2510.00328

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The widespread use of AI-assisted pair programming can inject a larger volume of difficult-to-maintain code into software projects, accelerating the accumulation of technical debt.

    “The widespread use of AI-assisted pair programming - and, in extreme cases, 'vibe coding' - can inject a larger volume of difficult-to-maintain code (Pimenova et al. 2025, Fawzy et al. 2025) into software projects, accelerating the accumulation of technical debt.”

    From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Fawzy et al. 2025

    Section 4.3 (p. 8) reports that vibe coders may keep rapidly generated, flawed code and recognize that it creates technical debt over time. Section 3.7 (p. 6) also describes some generated code as poorly structured and hard to extend. The paper does not establish that widespread AI-assisted pair programming injects a larger volume of such code into software projects; its findings concern vibe coding practices. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Vibe coding enables rapid progress and is often perceived as highly efficient by its users, but the resulting code is frequently described as fast but flawed.

    “A recurring theme is a speed-quality paradox (Fawzy et al. 2025): although vibe coding enables rapid progress and is often perceived as highly efficient by its users, the resulting code is frequently described as fast but flawed.”

    From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Fawzy et al. 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 15

Research works printed but not linked: 31

  1. [4][4] Jean Adams, Frances C. Hillier-Brown, Helen J. Moore, Amelia A. Lake, Vera Araujo-Soares, Martin White, and Carolyn Summerbell. 2016. Searching and synthesising 'grey literature' and 'grey information' in public health: critical reflections on three case studies. Systematic Reviews 5, 1 (2016), 164. https: //doi.org/10.1186/s13643-016-0337-y - · DOI · Paper page
  2. [5][5] Richard J. Adams, Paul Smart, and Anne S. Huff. 2016. Searching for grey literature for systematic reviews: challenges and benefits. Research Synthesis Methods 7, 3 (2016), 263-281. https://doi.org/10.1002/jrsm.1106 - · DOI · Paper page
  3. [6][6] Shraddha Barke, Michael B James, and Nadia Polikarpova. 2023. Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (2023), 85-111. -
  4. [8][8] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77. -
  5. [9][9] Simon Briscoe and Morwenna Rogers. 2024. An alternative screening approach for Google Search identifies an accurate and manageable number of results for a systematic review (case study). Health Information & Libraries Journal 41, 2 (2024), 149-155. -
  6. [11][11] Domenico Cotroneo, Cristina Improta, and Pietro Liguori. 2025. Human-Written vs. AI-Generated Code: A Large-Scale Study of Defects, Vulnerabilities, and Complexity. arXiv preprint arXiv:2508.21634 (2025). - · arXiv
  7. [14][14] Molly Q Feldman and Carolyn Jane Anderson. 2024. Non-expert programmers in the generative AI future. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work . 1-19. -
  8. [15][15] Samuel Ferino, Rashina Hoda, John Grundy, and Christoph Treude. 2025. Junior Software Developers' Perspectives on Adopting LLMs for Software Engineering: a Systematic Literature Review. arXiv preprint arXiv:2503.07556 (2025). - · arXiv
  9. [16][16] Yujia Fu, Peng Liang, Amjed Tahir, Zengyang Li, Mojtaba Shahin, Jiaxin Yu, and Jinfu Chen. 2025. Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study. ACM Transactions on Software Engineering and Methodology (2025). -
  10. [17][17] Vahid Garousi, Michael Felderer, and Mika V Mäntylä. 2019. Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and software technology 106 (2019), 101-121. -
  11. [20][20] Philipp Haindl and Gerald Weinberger. 2024. Does ChatGPT help novice programmers write better code? Results from static code analysis. IEEE Access (2024). -
  12. [24][24] Fernando Kamei, Igor Wiese, Crescencio Lima, Ivanilton Polato, Vilmar Nepomuceno, Waldemar Ferreira, Márcio Ribeiro, Carolline Pena, Bruno Cartaxo, Gustavo Pinto, and Sérgio Soares. 2021. Grey Literature in Software Engineering: A critical review. Information and Software Technology 138 (2021), 106609. https://doi.org/10.1016/j.infsof.2021.106609 - · DOI
  13. [25][25] Fernando Kenji Kamei. 2019. The use of grey literature review as evidence for practitioners. ACM SIGSOFT Software Engineering Notes 44, 3 (2019), 23-23. -
  14. [28][28] Syed Mohammad Kashif, Peng Liang, and Amjed Tahir. 2025. On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices. arXiv preprint arXiv:2504.16485 (2025). - · arXiv
  15. [30][30] Hans-Alexander Kruse, Tim Puhlfurb, and Walid Maalej. 2024. Can Developers Prompt? A Controlled Experiment for Code Documentation Generation. In 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE Computer Society, 574-586. -
  16. [31][31] MohammadAminKuhail, Sujith Samuel Mathew, Ashraf Khalil, Jose Berengueres, and Syed Jawad Hussain Shah. 2024. 'Will I be replaced?' Assessing ChatGPT's effect on software development and programmer perceptions of AI tools. Science of Computer Programming 235 (2024), 103111. -
  17. [33][33] Sherlock A Licorish, Ansh Bajpai, Chetan Arora, Fanyu Wang, and Kla Tantithamthavorn. 2025. Comparing Human and LLM Generated Code: The Jury is Still Out! arXiv preprint arXiv:2501.16857 (2025). - · arXiv
  18. [34][34] Vahid Majdinasab, Michael Joshua Bishop, Shawn Rasheed, Arghavan Moradidakhel, Amjed Tahir, and Foutse Khomh. 2024. Assessing the security of github copilot's generated code-a targeted replication study. In 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 435-444. -
  19. [45][45] Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2025. Asleep at the keyboard? assessing the security of github copilot's code contributions. Commun. ACM 68, 2 (2025), 96-105. -
  20. [46][46] Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of AI on developer productivity: Evidence from Github Copilot. arXiv preprint arXiv:2302.06590 (2023). - · arXiv
  21. [47][47] James Prather, Brent N Reeves, Paul Denny, Brett A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. 'It's weird that it knows what i want': Usability and interactions with copilot for novice programmers. ACM transactions on computer-human interaction 31, 1 (2023), 1-31. -
  22. [48][48] James Prather, Brent N Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S Randrianasolo, Brett A Becker, Bailey Kimmel, Jared Wright, and Ben Briggs. 2024. The widening gap: The benefits and harms of generative ai for novice programmers. In Proceedings of the 2024 ACM Conference on International Computing Education Research-Volume 1 . 469-486. -
  23. [53][53] Andreas Scholl and Natalie Kiesler. 2024. How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course. arXiv preprint arXiv:2407.20792 (2024). - · arXiv
  24. [56][56] Judy Sheard, Paul Denny, Arto Hellas, Juho Leinonen, Lauri Malmi, and Simon. 2024. Instructor Perceptions of AI Code Generation Tools-A Multi-Institutional Interview Study. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 . 1223-1229. -
  25. [59][59] Jacopo Soldani, Damian Andrew Tamburri, and Willem-Jan Van Den Heuvel. 2018. The pains and gains of microservices: A systematic grey literature review. Journal of Systems and Software 146 (2018), 215-232. -
  26. [65][65] Amjed Tahir, Shawn Rasheed, Jens Dietrich, Negar Hashemi, and Lu Zhang. 2023. Test flakiness' causes, detection, impact and responses: A multivocal review. Journal of Systems and Software 206 (2023), 111837. -
  27. [66][66] Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. 2022. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Chi conference on human factors in computing systems extended abstracts . 1-7. -
  28. [70][70] Justin D Weisz, Shraddha Vijay Kumar, Michael Muller, Karen-Ellen Browne, Arielle Goldberg, Katrin Ellice Heintze, and Shagun Bajpai. 2025. Examining the use and impact of an ai code assistant on developer productivity and experience in the enterprise. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems . 1-13. -
  29. [77][77] Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, Anders Wesslén, et al. 2012. Experimentation in software engineering . Vol. 236. Springer. -
  30. [79][79] Yangtian Zi, Luisa Li, Arjun Guha, Carolyn Anderson, and Molly Q Feldman. 2025. 'I Would Have Written My Code Differently': Beginners Struggle to Understand LLM-Generated Code. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering . 1479-1488. -
  31. [80][80] Rina Zviel-Girshin. 2024. The Good and Bad of AI Tools in Novice Programming Education. Education Sciences 14, 10 (2024), 1089.

Authors
Fawzy, A., Tahir, A., & Blincoe, K.
Affiliations
  • Massey University (NZ)
  • University of Auckland (NZ)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
212-223
DOI
10.48550/arxiv.2510.00328
arXiv
2510.00328
OpenAlex
W4414806416
Cited by
15
References
0
Citations by year
2026: 15
FWCI
160.22
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
Keywords
vibe coding, AI-assisted code generation, large language models, developer practices, quality assurance, speed-quality trade-off, developer motivation, software maintainability, AI-assisted software development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review