CoChat

The Future of AI-Driven Software Engineering

Open access
Terragni, Valerio, Vella, Annie, Roop, Partha, Blincoe, Kelly · 2025 · ACM Transactions on Software Engineering and Methodology

The DOI matches its Crossref record; 112 of the 112 references it cites were found in a registry or an index.

Abstract

A paradigm shift is underway in Software Engineering, with AI systems such as LLMs playing an increasingly important role in boosting software development productivity. This trend is anticipated to persist. In the next years, we expect a growing symbiotic partnership between human software developers and AI. The Software Engineering research community cannot afford to overlook this trend; we must address the key research challenges posed by the integration of AI into the software development process. In this article, we present our vision of the future of software development in an AI-driven world and explore the key challenges that our research community should address to realize this vision.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3715003

Cites

112 of 112 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Logical architecture of the envisioned future symbiosis of Software Engineers and AI
Figure 1 Logical architecture of the envisioned future symbiosis of Software Engineers and AICC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 16

  1. Vivek Basanagoudar and Abhijay Srekanth. 2023. Copyright conundrums in generative AI: Github Copilot’s not-so-fair use of open-source licensed code. Journal of Intellectual Property Studies 7 (2023), 58–68.
  2. Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https://arxiv.org/abs/2107.03374 · arXiv
  3. T. Y. Chen, S. C. Cheung, and S. M. Yiu. 1998. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report. Technical Report HKUST-CS98-01, Department of Computer Science, The Hong Kong University of Science and Technology.
  4. Wenpin Hou and Zhicheng Ji. 2024. A systematic evaluation of large language models for generating programming code. arXiv:2403.00894. Retrieved from https://arxiv.org/abs/2403.00894 · arXiv
  5. Dong Huang Qingwen Bu Jie Zhang Xiaofei Xie Junjie Chen and Heming Cui. 2023. Bias assessment and mitigation in LLM-based code generation. arXiv:2309.14345. Retrieved from https://arxiv.org/abs/2309.14345 · arXiv
  6. Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, and Kelly Blincoe. 2024. Understanding the impact of APIs behavioral breaking changes on client applications. In Proceedings of the 1st ACM on Software Engineering (PACMSE), 1238–1261.
  7. Michael Konstantinou Renzo Degiovanni and Mike Papadakis. 2024. Do LLMs generate test oracles that capture the actual or the expected program behaviour? arXiv:2410.21136. Retrieved from https://arxiv.org/abs/2410.21136 · arXiv
  8. Sebastian Lubos, Alexander Felfernig, Thi Ngoc Trang Tran, Damian Garber, Merfat El Mansi, Seda Polat Erdeniz, and Viet-Man Le. 2024. Leveraging LLMs for the quality assurance of software requirements. In Proceedings of the 2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, 389–397.
  9. Quang-Hung Luu Huai Liu and Tsong Yueh Chen. 2023. Can ChatGPT advance software testing intelligence? An experience report on metamorphic testing. arXiv:2310.19204. Retrieved from https://arxiv.org/abs/2310.19204 · arXiv
  10. Garima Malik Mucahit Cevik Devang Parikh and Ayse Basar. 2024. Supervised semantic similarity-based conflict detection algorithm: S3CDA. arXiv:2206.13690. Retrieved from https://arxiv.org/abs/2206.13690 · arXiv
  11. Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. 2023. From words to watts: Benchmarking the energy costs of large language model inference. In Proceedings of the 2023 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1–9.
  12. Seung Yeob Shin Fabrizio Pastore Domenico Bianculli and Alexandra Baicoianu. 2024. Towards generating executable metamorphic relations using large language models. arXiv:2401.17019. Retrieved from https://arxiv.org/abs/2401.17019 · arXiv
  13. Christos Tsigkanos, Pooja Rani, Sebastian Müller, and Timo Kehrer. 2023. Large language models: The next frontier for variable discovery within metamorphic testing? In Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 678–682.
  14. Simon Van Oordt and Emitza Guzman. 2021. On the role of user feedback in software evolution: A practitioners’ perspective. In Proceedings of the 2021 IEEE 29th International Requirements Engineering Conference (RE). IEEE, 221–232.
  15. Zhuokui Xie Yinghao Chen Chen Zhi Shuiguang Deng and Jianwei Yin. 2023. ChatUniTest: A ChatGPT-based automated unit test generation tool. arXiv:2305.04764. Retrieved from https://arxiv.org/abs/2305.04764 · arXiv
  16. Burak Yetiştiren Işık Özsoy Miray Ayerdem and Eray Tüzün. 2023. Evaluating the code quality of AI-assisted code generation tools: An empirical study on GitHub Copilot Amazon CodeWhisperer and ChatGPT. arXiv:2304.10778. Retrieved from https://arxiv.org/abs/2304.10778 · arXiv

Cited by

See all 63

Metadata

Authors
Terragni, V., Vella, A., Roop, P., & Blincoe, K.
Affiliations
University of Auckland (NZ)
Venue
ACM Transactions on Software Engineering and Methodology
Published
2025-05-26
Volume
34
Issue
5
Pages
1-20
ISSN
  • 1049-331X (ISSN-L)
  • 1557-7392
OpenAlex
W4406798194
Cited by
62
References
97
Citations by year
2026: 33 · 2025: 29
FWCI
65.00
Type
journal article
Language
en
Open access
Locations
  • ACM Transactions on Software Engineering and Methodology (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Marsden Fund Council
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
Keywords
large language models, human-AI collaboration, software productivity, future of software development, research challenges

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.