The DOI matches its Crossref record; 235 of the 235 references it cites were found in a registry or an index.
Generative Artificial Intelligence for Software Engineering—A Research Agenda
Open accessAbstract
Context: Generative artificial intelligence (GenAI) tools have become increasingly prevalent in software development, offering assistance to various managerial and technical project activities. Notable examples of these tools include OpenAI's ChatGPT, GitHub Copilot, and Amazon CodeWhisperer. Objective: Although many recent publications have explored and evaluated the application of GenAI, a comprehensive understanding of the current development, applications, limitations, and open challenges remains unclear to many. Particularly, we do not have an overall picture of the current state of GenAI technology in practical software engineering usage scenarios. Method: We conducted a literature review and focus groups for a duration of five months to develop a research agenda on GenAI for software engineering. Results: We identified 78 open research questions (RQs) in 11 areas of software engineering. Our results show that it is possible to explore the adoption of GenAI in partial automation and support decision‐making in all software development activities. While the current literature is skewed toward software implementation, quality assurance and software maintenance, other areas, such as requirements engineering, software design, and software engineering education, would need further research attention. Common considerations when implementing GenAI include industry‐level assessment, dependability and accuracy, data accessibility, transparency, and sustainability aspects associated with the technology. Conclusions: GenAI is bringing significant changes to the field of software engineering. Nevertheless, the state of research on the topic still remains immature. We believe that this research agenda holds significance and practical value for informing both researchers and practitioners about current applications and guiding future research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“LLM capabilities have been associated with their size [263], resulting in the rapid growth of model size [264].”
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“These frameworks should be designed with input from diverse stakeholders, including ethicists, technologists, and affected communities, to ensure that they address a broad range of ethical concerns [269].”
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Research works printed but not linked: 52
- S.Ouyang J. M.Zhang M.Harman andM.Wang “LLM is Like a Box of Chocolates: The Non‐Determinism of ChatGPT in Code Generation ”2023 arXiv:2308.02828 [cs] https://doi.org/10.48550/arXiv.2308.02828. · DOI · Paper page
- 10.48550/arXiv.2409.06741 · DOI
- Brown T. B., 2020, Proceedings of the 34th International Conference on Neural Information Processing Systems (Red Hook, NY, USA, 2020‐12‐06) (NIPS'20), 1877
- A.RadfordandK.Narasimhan “Improving Language Understanding by Generative Pre‐Training ”2018 https://www.semanticscholar.org/paper/Improving‐Language‐Understanding‐by‐Generative‐Radford‐Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035.
- Heafield K., 2013, Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 690
- Radford A., 2019, Language Models Are Unsupervised Multitask Learners, OpenAI Blog, 1, 9
- Q.Zhang M.Chen A.Bukharin et al. “Adaptive Budget Allocation for Parameter‐Efficient Fine‐Tuning ”2023 arXiv:2303.10512 [cs.CL]. · arXiv
- J.Zhang Y.Chen N.Niu andC.Liu “A Preliminary Evaluation of ChatGPT in Requirements Information Retrieval ”2023 arXiv Preprint arXiv:2304.12562. · DOI
- Ezzini S., 2023, Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 1277
- Luo X., 2023, Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (Rochester, MI, USA) (ASE ‘22)
- Chen B., 2023, MoDRE Workshop at Requirement Engineering Conference
- Nahar N., 2016, Proceedings of the 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV), 35
- Ahmad A., 2023, Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering (Oulu, Finland) (EASE ‘23), New York, NY, 279
- Maranhão Junior J. J., 2023, AI‐Assisted Agile Software Development Workshop at XP2023 Conference
- T.StojanovicandS. D.Lazarević “The Application of ChatGPT for Identification of Microservices ”3 no.1(2023):99–105 Number: 1 https://ebt.rs/journals/index.php/conf‐proc/article/view/181.
- Gamma E., 1995, Design Patterns
- Viviani L., 2023, International Conference on Agile Software Development, 77
- 10.1109/ICSE-Companion55297.2022.9793778 · DOI · Paper page
- Siddiq M. L., 2023, Proceedings of the 2023 IEEE/ACM 2nd International Workshop on Natural Language‐Based Software Engineering (NLBSE) (2023–05), 56
- Y.Dong J.Xue Z.Jin andG.Li “Self‐Collaboration Code Generation via ChatGPT ”2023 arXiv:2304.07590 [cs.SE]. · arXiv
- Jiao Sun Q., 2022, Proeedings of the 27th International Conference on Intelligent User Interfaces (New York, NY, USA, 2022‐03‐22) (IUI ‘22), 212
- H.Li H.Yu Y.Zhai andZ.Qian “The Hitchhiker's Guide to Program Analysis: A Journey With Large Language Models ”2023 arXiv:2308.00245 [cs.SE]. · arXiv
- Salza P., 2021, On the Effectiveness of Transfer Learning for Code Search, 49, no. 4, 1804
- Wei M., 2022, Proceedings of the 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) (2022–05), 376
- Mastropaolo A., 2021, Proceedings of the 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 336
- X.Liu Y.Zheng D.Zhengxiao et al. “GPT Understands Too ”(2021) arXiv:2103.10385 [cs.CL]. · arXiv
- R.Li L. B.Allal Y.Zi et al. “StarCoder: May the Source be With You!”(2023) arXiv:2305.06161 [cs.CL]. · arXiv
- Planning S., 2002, The Economic Impacts of Inadequate Infrastructure for Software Testing, National Institute of Standards and Technology, 1
- 10.1145/3572905 · DOI
- W.Ma S.Liu W.Wang et al. “The Scope of ChatGPT in Software Engineering: A Thorough Investigation ”(2023) arXiv Preprint arXiv:2305.12138. · arXiv
- J.White S.Hays F.Quchen J.Spencer‐Smith andD. C.Schmidt “Chatgpt Prompt Patterns for Improving Code Quality Refactoring Requirements Elicitation and Software Design ”(2023) arXiv Preprint arXiv:2303.07839. · DOI
- R.Feldt S.Kang J.Yoon andS.Yoo “Towards Autonomous Testing Agents via Conversational Large Language Models ”(2023) arXiv Preprint arXiv:2306.05152. · DOI · Paper page
- W.Sun C.Fang Y.You et al. “Automatic Code Summarization via ChatGPT: How Far Are We?”(2023) arXiv:2305.12865 [cs.SE]. · arXiv
- Ahmed T., 2023, Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (Rochester, MI, USA) (ASE'22)
- Q.Zhang C.Fang W.Sun et al. “Boosting Automated Patch Correctness Prediction via Pre‐Trained Language Model ”(2023) arXiv:2301.12453 [cs.SE]. · arXiv
- X.Hou Y.Zhao Y.Liu et al. “Large Language Models for Software Engineering: A Systematic Literature Review ”(2023) arXiv:2308.10620 [cs.SE]. · arXiv
- Sommerville I., 2016, Software Engineering
- Weisz J. D., 2021, Proceedings of the 26th International Conference on Intelligent User Interfaces (New York, NY, USA, 2021‐04‐14) (IUI ‘21), 402
- Steven I., 2023, Proceedings of the 28th International Conference on Intelligent User Interfaces (New York, NY, USA, 2023‐03‐27) (IUI ‘23), 491
- Anthony P., 2002, Ammeter and Janet M Dukerich. 2002. Leadership, Team Building, and Team Member Characteristics in High Performance Project Teams, Engineering Management Journal, 14, 3
- Dam H. K., 2019, Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE‐NIER), 41
- Haytham H., 2020, Elmousalami. 2020. Comparison of Artificial Intelligence Techniques for Project Conceptual Cost Prediction: A Case Study and Comparative Analysis, IEEE Transactions on Engineering Management, 68, 183
- OpenAI “GPT‐4 Technical Report ”(2023) arXiv:2303.08774 [cs.CL]. · arXiv
- D.Adiwardana M.‐T.Luong D. R.So et al. “Towards a Human‐Like Open‐Domain Chatbot. CoRR Abs/2001.09977 ”(2020) arXiv:2001.09977 https://arxiv.org/abs/2001.09977. · arXiv
- European Commission, 2021, EUR‐Lex ‐ 52021PC0206 ‐ EN ‐ EUR‐Lex
- Binkis M., 2021, Information and Software Technologies (Cham, 2021) (Communications in Computer and Information Science), 367
- Berrezueta‐Guzman J., 2003, Proceedings of the 2023 IEEE 35th International Conference on Software Engineering Education and Training (CSEE&T) (2023–08), 147
- Kazemitabaar M., 2024, Proceedings of the 23rd Koli Calling International Conference on Computing Education Research (Koli Calling ‘23), 1
- Balse R., 2023, Proceedings of the 16th Annual ACM India Compute Conference (COMPUTE ‘23), 49
- 10.48550/arXiv.2304.12269 · DOI
- W.Ma S.Liu W.Wang et al. “The Scope of ChatGPT in Software Engineering: A Thorough Investigation ”2023 arXiv:2305.12138 [cs] https://doi.org/10.48550/arXiv.2305.12138. · DOI
- F.Liu L.Yang S.Lin et al. “Exploring and Evaluating Hallucinations in LLM‐Powered Code Generation ”2024 arXiv:2404.00971 [cs] https://doi.org/10.48550/arXiv.2404.00971. · DOI · Paper page
Cited by
Sorted by citations
LLM-Based Agents for Automating the Enhancement of User Story Quality: An Early Report
Zhang, Z., Rayhan, M., Herda, T., Goisauf, M., & Abrahamsson, P. (2024) · Lecture notes in business information processing · doi:10.1007/978-3-031-61154-4_8 · Cited by 67
Self-citationBano, M., Hoda, R., Zowghi, D., & Treude, C. (2023) · Automated Software Engineering · doi:10.1007/s10515-023-00407-8 · Cited by 63
Generative AI Meets Knowledge Management: Insights From Software Development Practices
Kirchner, K., Bolisani, E., Kassaneh, T. C., Scarso, E., & Taraghi, N. (2025) · Knowledge and Process Management · doi:10.1002/kpm.70004 · Cited by 40
Gadde, A. (2025) · Journal of Computer Science and Technology Studies · doi:10.32996/jcsts.2025.7.4.66 · Cited by 31
A Review on Vibe Coding: Fundamentals, State-of-the-art, Challenges and Future DirectionsPreprint
Ray, P. P. (2025) · doi:10.36227/techrxiv.174681482.27435614/v1 · Cited by 27
Metadata
- Authors
- Nguyen‐Duc, A., Cabrero‐Daniel, B., Przybylek, A., Arora, C., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K., Saari, M., Zhang, Z., Le, H., Quan, T., & Abrahamsson, P.
- Affiliations
- Norwegian University of Science and Technology (NO)
- University of Gothenburg (SE)
- Gdansk University of Technology Gdansk Poland
- Monash University (AU)
- Free University of Bozen-Bolzano (IT)
- Austrian Post—Konzern IT Vienna Austria
- University of Helsinki (FI)
- Tampere University (FI)
- Vietnam National University Ho Chi Minh City (VN)
- Corresponding author
- Nguyen‐Duc, Anh
- ORCID
- Venue
- Software: Practice and Experience
- Publisher
- Wiley
- Published
- 2025-07-29
- Volume
- 55
- Issue
- 11
- Pages
- 1806-1843
- ISSN
- 0038-0644 (ISSN-L)
- 1097-024X
- OpenAlex
- W4412899734
- Cited by
- 60
- References
- 222
- Citations by year
- 2026: 47 · 2025: 13
- FWCI
- 62.39
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- Software Practice and Experience (publishedVersion)
- Työväentutkimus Vuosikirja (submittedVersion, other-oa)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- Open Research (University of Surrey) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- generative AI, software engineering, research agenda, open research questions, ChatGPT, GitHub Copilot, software development automation, decision support, requirements engineering, software design, software engineering education, software maintenance, quality assurance, dependability, transparency, sustainability, data accessibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
