Towards Human-Bot Collaborative Software Architecting with ChatGPT
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Architecting software-intensive systems can be a complex process. It deals with the daunting tasks of unifying stakeholders’ perspectives, designers’ intellect, tool-based automation, pattern-driven reuse, and so on, to sketch a blueprint that guides software implementation and evaluation. Despite its benefits, architecture-centric software engineering (ACSE) suffers from a multitude of challenges. ACSE challenges could stem from a lack of standardized processes, socio-technical limitations, and scarcity of human expertise etc. that can impede the development of existing and emergent classes of software. Software Development Bots (DevBots) trained on large language models can help synergise architects’ knowledge with artificially intelligent decision support to enable rapid architecting in a human-bot collaborative ACSE. An emerging solution to enable this collaboration is ChatGPT, a disruptive technology not primarily introduced for software engineering, but is capable of articulating and refining architectural artifacts based on natural language processing. We detail a case study that involves collaboration between a novice software architect and ChatGPT to architect a service-based software. Future research focuses on harnessing empirical evidence about architects’ productivity and explores socio-technical aspects of architecting with ChatGPT to tackle challenges of ACSE.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The authors also uncovered ChatGPT’s capacity to identify and address ethical, governance, and socio-technical considerations within this context66.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [66]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This has resulted in a number of recent studies that investigate the impact of technology on design and creative workflows, and the potential for its usage as a design support tool (e.g.[2,11,20,24,58]).”
From Is Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism (2024) · cites this paper as [2]
In the citing paper
“An initial case study within this domain explored the potential synergies between ChatGPT and architects’ decision-making, aiming to automate various facets of ACSE, as documented by Ahmad66.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [66]
In the citing paper
“A notable example of collaborative programming involves the concept of Human-Bot Collaborative Architecting, with a specific emphasis on the integration of ChatGPT into human decision-making processes to streamline aspects of Architecture-Centric Software Engineering (ACSE).”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [66]
In the citing paper
“Consistency and ethics: Studies on Architecture-Centric Software Engineering (ACSE) using ChatGPT highlight several concerns66.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [66]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A survey on software architecture analysis methods
Dobrica, L., & Niemela, E. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019479 · Cited by 518 · cited 8×
“To design and conduct this study, and considering the internal validity, we followed a systematic approach and utilized a well-known architecting process [9] and architecture evaluation method [6].”
BackgroundMethodInfluentialA Categorical Archive of ChatGPT Failures
Borji, A. (2023) · arXiv · doi:10.48550/arxiv.2302.03494 · Cited by 514 · cited 1×
“However, it remains an unexplored territory in terms of its potential and perils in the context of software development processes [17], [18].”
BackgroundAn empirical evaluation of GitHub copilot's code suggestions
Nguyen, N., & Nadi, S. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528470 · Cited by 313 · cited 3×
“Software Development Bots (DevBots) represent conversational agents or recommender systems, driven by AI, to assist software engineers by offering certain degree of automation and/or inducing intelligence in software engineering process [14, 15].”
BackgroundMethodA general model of software architecture design derived from five industrial approaches
Hofmeister, C., Kruchten, P., Nord, R. L., Obbink, H., Ran, A., & America, P. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.05.024 · Cited by 307
What Industry Needs from Architectural Languages: A Survey
Malavolta, I., Lago, P., Muccini, H., Pelliccione, P., & Tang, A. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.74 · Cited by 296 · cited 3×
“The ASRs are synthesized into an architectural model that can be expressed with an architectural (modeling) language, like UML or other architectural languages [14].”
BackgroundMethodInfluentialThe Past, Present, and Future for Software Architecture
Kruchten, P., Obbink, H., & Stafford, J. (2006) · IEEE Software · doi:10.1109/ms.2006.59 · Cited by 231
Applications of AI in classical software engineering
Barenkamp, M., Rebstadt, J., & Thomas, O. (2020) · AI Perspectives · doi:10.1186/s42467-020-00005-4 · Cited by 123 · cited 3×
“The research on synergizing AI and SE can be classified into two distinct dimensions namely AI for SE (artificial intelligence in software engineering) and SE for AI (software engineering for artificial intelligence) [5] [6].”
BackgroundSoftware Architecture in a Changing World
Woods, E. (2016) · IEEE Software · doi:10.1109/ms.2016.149 · Cited by 93 · cited 1×
“Specifically, SE solutions need to maintain the so-called ‘intelligence equilibrium’ – i.e., unifying and balancing machine intelligence and human intellect – in processes, patterns, and tools etc. for emergent classes of software, such as blockchain and quantum applications [16].”
BackgroundCollaborative Modeling and Group Decision Making Using Chatbots in Social Networks
Perez-Soler, S., Guerra, E., & de Lara, J. (2018) · IEEE Software · doi:10.1109/ms.2018.290101511 · Cited by 76 · cited 2×
“[14] (investigating Github Copilot) and a tool (Chatbot named SOCIO) developed by Pérez-Soler et al.[15] highlight the growing impacts and emerging challenges of using bots in software modeling and programming.”
BackgroundMethodAn Analysis of the Automatic Bug Fixing Performance of ChatGPT
(2023) · doi:10.48550/arxiv.2301.08653 · Cited by 49 · cited 6×
“ChatGPT is not specifically developed to address software engineering challenges, however; it is well capable of generating versatile textual specifications including architectural requirements, Unified Modeling Language (UML) scripts, source code libraries, and test cases [11, 17].”
BackgroundMethodInfluential
Research works printed but not linked: 5
- 2. Anonymous. 2023. Replication Package: Towards Human-Bot Collaborative Software Architecting with ChatGPT. In https:// anonymous.4open.science/ r/ ChatGPT4SA-2B61.
- 3. Laurent Avila-Chauvet, Diana Mejía, and Christian Oswaldo Acosta Quiroz. 2023. ChatGPT as a Support Tool for Online Behavioral Task Programming. SSRN preprint SSRN:4329020 (2023).
- 8. Sebastian Herold, Christoph Knieke, Mirco Schindler, and Andreas Rausch. 2020. Towards Improving Software Architecture Degradation Mitigation by Machine Learning. In Proceedings of the 12th International Conference on Adaptive and Self-Adaptive Systems and Applications (ADAPTIVE). IARIA, 36-39.
- 11. Sajed Jalil, Suzzana Rafi, Thomas D LaToza, Kevin Moran, and Wing Lam. 2023. ChatGPT and Software Testing Education: Promises & Perils. arXiv preprint arXiv:2302.03287 (2023). · arXiv
- 16. Junaid Qadir. 2022. Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education. TechRxiv preprint techrxiv.21789434 (2022).
Cited by
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01852 · Cited by 1,172
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
“Ahmad et al. [1] proposed a method for human-machine collaboration using ChatGPT to create software architecture.”
MethodLarge language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
“Ahmad et al. [202] explore the way in which a novice software architect could interact with ChatGPT.”
BackgroundUnlocking the opportunities through ChatGPT Tool towards ameliorating the education system
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100115 · Cited by 535
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
“Ahmad et al. [202] explore the way in which a novice software architect could interact with ChatGPT.”
BackgroundAl Naqbi, H., Bahroun, Z., & Ahmed, V. (2024) · Sustainability · doi:10.3390/su16031166 · Cited by 302
VeriGen: A Large Language Model for Verilog Code Generation
Thakur, S., Ahmad, B., Pearce, H., Tan, B., Dolan-Gavitt, B., Karri, R., & Garg, S. (2024) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3643681 · Cited by 228
“While considerable effort has been undertaken which explores the training and utilization of language models for writing software [1, 5, 14] as well as in their down-stream effects (e.”
BackgroundChip-Chat: Challenges and Opportunities in Conversational Hardware Design
Blocklove, J., Garg, S., Karri, R., & Pearce, H. (2023) · 2023 ACM/IEEE 5th Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad58807.2023.10299874 · Cited by 216
“Similar studies are being carried out for ChatGPT in various domains, including software [11] and education [12], making this investigation into the impact of conversational LLMs on hardware design both relevant and timely.”
BackgroundSohail, S. S., Farhat, F., Himeur, Y., Nadeem, M., Madsen, D. Ø., Singh, Y., Atalla, S., & Mansoor, W. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101675 · Cited by 192
“Ahmad et al. (2023) used ChatGPT for analyzing, synthesizing, and evaluating the architecture of a services-oriented software application.”
Role of activity-based learning and ChatGPT on students' performance in education
Al Shloul, T., Mazhar, T., Abbas, Q., Iqbal, M., Ghadi, Y. Y., Shahzad, T., Mallek, F., & Hamam, H. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100219 · Cited by 173
“It has been determined that ChatGPT cannot complete a software testing training program independently (Ahmad et al., 2023a; Zhang et al., 2023).”
BackgroundTowards an understanding of large language models in software engineering tasks
Zheng, Z., Ning, K., Zhong, Q., Chen, J., Chen, W., Guo, L., Wang, W., & Wang, Y. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10602-0 · Cited by 116
“…et al. 2021; Li et al. 2022c; Chai et al. 2022; Wang et al. 2023e), many research works focused on the application of LLMs in the software engineering domain (Ahmad et al. 2023a; Zhang et al. 2023b; Li et al. 2023a; Barke et al. 2023a; Zhuo et al. 2023; Gozalo-Brizuela and Garrido-Merchan 2023).”
ChatGPT FOR PROGRAMMING NUMERICAL METHODS
Kashefi, A., & Mukerji, T. (2023) · Journal of Machine Learning for Modeling and Computing · doi:10.1615/jmachlearnmodelcomput.2023048492 · Cited by 100
Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice
Khojah, R., Mohamad, M., Leitner, P., & de Oliveira Neto, F. G. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660788 · Cited by 86
“Recent studies have been looking into the helpfulness of chatbots in terms of decision-making [17] and efficiency [18], which can vary depending on the context.”
BackgroundInvestigating ChatGPT’s Potential to Assist in Requirements Elicitation Processes
Ronanki, K., Berger, C., & Horkoff, J. (2023) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa60479.2023.00061 · Cited by 159
“Ahmed et al. [3] investigate the potential and limitations of using ChatGPT to assist an architect in Architecture-centric Software Engineering (ACSE) using a Human-DevBot collaboration approach.”
MethodGenerative Artificial Intelligence for Software Engineering - a Research Agenda
NguyenDuc, A., Cabrero-Daniel, B., Arora, C., Przybylek, A., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K.-K., et al. (2023) · doi:10.2139/ssrn.4622517 · Cited by 71
Software Engineering Education Must Adapt and Evolve for an LLM Environment
Kirova, V. D., Ku, C. S., Laracy, J. R., & Marlowe, T. J. (2024) · Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3626252.3630927 · Cited by 67
“Educators have explored ways to incorporate AI into various educational disciplines [34], with computer science and software engineering curricula being no exception [1, 3].”
BackgroundOn the use of large language models in model-driven engineering
Rocco, J. D., Ruscio, D. D., Sipio, C. D., Nguyen, P. T., & Rubei, R. (2025) · Software & Systems Modeling · doi:10.1007/s10270-025-01263-8 · Cited by 111
“Ahmand et al. [3] applied ChatGPT to support the generation of software architecture from textual requirements.”
Quantum Cryptography for Future Networks Security: A Systematic Review
Durr-E-Shahwar, Imran, M., Altamimi, A. B., Khan, W., Hussain, S., & Alsaffar, M. S. (2024) · IEEE Access · doi:10.1109/access.2024.3504815 · Cited by 114
“…review will offer insightful information about the process of switching from conventional to QC, giving data scientists, researchers, and practitioners thorough information on the state-of-the-art knowledge, difficulties, and potential solutions for enhancing network security in the future [21].”
Rane, N. L. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/npm3d · Cited by 71
Accelerating Software Development Using Generative AI: ChatGPT Case Study
Rajbhoj, A., Somase, A., Kulkarni, P., & Kulkarni, V. (2024) · Proceedings of the 17th Innovations in Software Engineering Conference · doi:10.1145/3641399.3641403 · Cited by 50
GPTSniffer: A CodeBERT-based classifier to detect source code written by ChatGPT
Nguyen, P. T., Rocco, J. D., Sipio, C. D., Rubei, R., Ruscio, D. D., & Penta, M. D. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112059 · Cited by 113
On opportunities and challenges of large multimodal foundation models in education
Küchemann, S., Avila, K. E., Dinc, Y., Hortmann, C., Revenga, N., Ruf, V., Stausberg, N., Steinert, S., Fischer, F., Fischer, M., et al. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00301-w · Cited by 47
“The authors also uncovered ChatGPT ’ s capacity to identify and address ethical, governance, and socio-technical considerations within this context 66 .”
How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts
Wang, B., Wang, C., Liang, P., Li, B., & Zeng, C. (2024) · 2024 IEEE International Conference on Software Services Engineering (SSE) · doi:10.1109/sse62657.2024.00046 · Cited by 62
“The case study reflects the factors that need to be considered in collaborative architecture design with ChatGPT, such as Variation in responses and artifacts, level of human decision support [3].”
MethodSelf-citationTrustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps
Lo, D. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00010 · Cited by 40
“While recent research has begun delving into less-explored areas, e.g., managing bug reports [96], designing software architecture [97], building navigation aids for software engineering Q&A sites [98], etc., there remains significant scope for broader exploration.”
BackgroundSoftware engineering education in the era of conversational AI: current trends and future directions
Sengul, C., Neykova, R., & Destefanis, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1436350 · Cited by 84
How to refactor this code? An exploratory study on developer-ChatGPT refactoring conversations
AlOmar, E. A., Venkatakrishnan, A., Mkaouer, M. W., Newman, C. D., & Ouni, A. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3645081 · Cited by 86
“The ability to harness massive amounts of information in multiple modalities allows LLMs to perform various tasks that would typically require human intervention [3, 4, 15, 20, 20, 22, 26, 27, 30, 34, 45].”
BackgroundHuman-AI collaboration in software engineering: lessons learned from a hands-on workshop
Hamza, M., Siemon, D., Akbar, M. A., & Rahman, T. (2024) · Proceedings of the 7th ACM/IEEE International Workshop on Software-intensive Business · doi:10.1145/3643690.3648236 · Cited by 38
“Similarly, Ahmad et al. [18] explore the integration of ChatGPT in software architecture processes.”
Software Engineering by and for Humans in an AI Era
Abrahão, S., Grundy, J., Pezzè, M., Storey, M.-A., & Tamburri, D. A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715111 · Cited by 37
“An SE team thus includes a mix of differently skilled and experienced humans and differently capable AI-powered agents.”
Rodriguez, D. V., Lawrence, K. S., González, J., Brandfield-Harvey, B., Xu, L., Tasneem, S., Levine, D. L., & Mann, D. (2024) · JMIR Human Factors · doi:10.2196/52885 · Cited by 54
Prompting GPT –4 to support automatic safety case generation
Sivakumar, M., Belle, A. B., Shan, J., & Shahandashti, K. K. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124653 · Cited by 61
Learning by teaching with ChatGPT : The effect of teachable ChatGPT agent on programming education
Chen, A., Wei, Y., Le, H., & Zhang, Y. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.70001 · Cited by 87
ChatGPT API: Brief overview and integration in Software Development
Paredes, C. M. G., Machuca, C. R., & Claudio, Y. M. S. (2023) · International Journal of Engineering Insights · doi:10.61961/injei.v1i1.7 · Cited by 36
ChatGPT as a Software Development Bot: A Project-Based Study
Waseem, M., Das, T., Ahmad, A., Liang, P., Fahmideh, M., & Mikkonen, T. (2024) · International Conference on Evaluation of Novel Approaches to Software Engineering · doi:10.5220/0012631600003687 · Cited by 76
“For example, AI-driven tools have been substantiated for enhancing requirement gathering through natural language processing capabilities and facilitating design phases (Ahmad et al., 2023).”
BackgroundMethodInfluentialSelf-citationKokol, P. (2024) · Information · doi:10.3390/info15060354 · Cited by 48
System Architects Are not Alone Anymore: Automatic System Modeling with AI
Apvrille, L., & Sultan, B. (2024) · International Conference on Model-Based Software and Systems Engineering (MODELSWARD) · doi:10.5220/0012320100003645 · Cited by 30
“In a more architecture process-oriented study, Ahmad et al. (Ahmad et al., 2023) present a comprehensive report detailing their use of ChatGPT for software architecture tasks, which include generating requirements, UML models, and evaluating the proposed architecture.”
BackgroundIs Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism
Boucher, J. D., Smith, G., & Telliel, Y. D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641889 · Cited by 30
“This has resulted in a number of recent studies that investigate the impact of technology on design and creative work-flows, and the potential for its usage as a design support tool (e.g. [2, 11, 20, 24, 58]).”
BackgroundCombining low-code development with ChatGPT to novel no-code approaches: A focus-group study
Martins, J., Branco, F., & Mamede, H. S. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200289 · Cited by 42
Codepori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
Rasheed, Z., Malik, S., Kemell, K.-K., Muhammad, W., Saari, M., Systä, K., & Abrahamsson, P. (2024) · doi:10.2139/ssrn.4979510 · Cited by 27
How Do Software Developers Use ChatGPT? An Exploratory Study on GitHub Pull Requests
Chouchen, M., Bessghaier, N., Begoug, M., Ouni, A., AlOmar, E. A., & Mkaouer, M. W. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3645084 · Cited by 43
“This integration of LLMs into software development workflows not only remodels the coding landscape, but also contributes to the wider impact of these models on daily life [20].”
BackgroundZhou, Y., & Chen, C. (2025) · Design and Artificial Intelligence · doi:10.1016/j.daai.2025.100017 · Cited by 45
Kumar, J., & Chimalakonda, S. (2024) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3661167.3661210 · Cited by 37
“Further, the evolution of LLMs with newer models such as GPT family [1, 31], T5 family [23, 48], etc led to multiple adoptions and validation works in SE.”
MethodExploring LLM-based Automated Repairing of Ansible Script in Edge-Cloud Infrastructures
Kwon, S., Lee, S., Kim, T., Ryu, D., & Baik, J. (2023) · Journal of Web Engineering · doi:10.13052/jwe1540-9589.2263 · Cited by 34
“4 While LLMs are typically utilized for general purposes, such as Natural Language Processing (NLP), their powerful capabilities have also been found in automating SE tasks [2, 9, 14, 16, 28].”
BackgroundMethodAI-Driven Consistency of SysML Diagrams
Sultan, B., & Apvrille, L. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3640310.3674079 · Cited by 21
“In direct link with this paper’s very subject is the work by Ahmad et al. [2], which provides an insightful overview of ChatGPT’s application in various software architecture tasks, including generating requirements, creating UML models, and evaluating these models.”
Rasheed, Z., Waseem, M., Ahmad, A., Kemell, K., Wang, X., NguyenDuc, A., & Abrahamsson, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5149683 · Cited by 27
Self-citationOrden-Mejía, M. A., Carvache‐Franco, M., Huertas, A., Carvache‐Franco, O., & Carvache‐Franco, W. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2425882 · Cited by 29
“Both platforms take advantage of Transformer Neural Network Architecture, an area of research that merits further exploration due to its distinctive characteristics and advantages (Ahmad et al., 2023).”
Küchemann, S., Avila, K. E., Dinc, Y., Hortmann, C., Revenga, N., Ruf, V., Stausberg, N., Steinert, S., Fischer, F., Fischer, M. R., et al. (2024) · doi:10.35542/osf.io/n7dvf · Cited by 19
Tapan-Broutin, M. S. (2023) · Yuksekogretim Dergisi · doi:10.53478/yuksekogretim.1347061 · Cited by 39
ECHO: An Approach to Enhance Use Case Quality Exploiting Large Language Models
Vito, G. D., Palomba, F., Gravino, C., Martino, S. D., & Ferrucci, F. (2023) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa60479.2023.00017 · Cited by 21
“Several studies have shown the effectiveness of LLMs in supporting software development tasks [17], [18] (i.e., code generation, bug fixing, and architectural analysis).”
BackgroundJohns, B., Carroll, K., Medina, C., Lewark, R., & Walliser, J. (2024) · INCOSE International Symposium · doi:10.1002/iis2.13201 · Cited by 23
ChatGPT for Operating Systems: Higher-Order Thinking in Focus
Kofahi, M. A., & Husain, A. (2025) · Journal of Information Technology Education Research · doi:10.28945/5382 · Cited by 26
“The poor pe rformance of ChatGPT in perform ing software testing and training was also revealed by Ahmad et al. (2023) and Zhang et al. (2023).”
Impacts of the Usage of Generative Artificial Intelligence on Software Development Process
Santos, P. D. O., Figueiredo, A. C., Moura, P., Diirr, B., Alvim, A. C. F., & dos Santos, R. P. (2024) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.1145/3658271.3658337 · Cited by 23
“O viés de código e informações [3, 16, 43] geradas pelas ferra-mentas ChatGPT e Copilot, a possibilidade de divulgação de dados privados [7] e o risco de violação de direitos autorais [3, 26] também são preocupações relatadas nos estudos.”
BackgroundLambiase, S., Catolino, G., Palomba, F., & Ferrucci, F. (2025) · ACM Computing Surveys · doi:10.1145/3704806 · Cited by 19
“Other studies propose bots to improve and speed up agile processes by automatizing the agile retrospective step by collecting metrics from different sources [A7, A46].”
InfluentialIs ChatGPT a Good English to Arabic Machine Translation Tool?
Banimelhem, O., & Amayreh, W. (2023) · International Conference on Information and Communication Systems (ICICS) · doi:10.1109/icics60529.2023.10330525 · Cited by 17
“ChatGPT is built on the transformer architecture, which was revealed in 2017 and has since become the dominant framework for NLP models [2].”
Quevedo, E., Abdelfattah, A. S., Rodriguez, A., Yero, J., & Cerny, T. (2024) · SN Computer Science · doi:10.1007/s42979-024-02664-0 · Cited by 15
“Additionally, Ahmad et al. [40] investigate the possibility of a collaborative architecting of microservice-based software between spe-SN Correctly prompting the system is core to effectively using chatbots like ChatGPT.”
TTool-AI: A Large Language Model-Based Assistant for Model Driven Engineering
Sultan, B., & Apvrille, L. (2025) · SN Computer Science · doi:10.1007/s42979-025-04444-w · Cited by 12
“…in this paper, extends TTool with an AI-based assistant that is intended to assist the TTool’s users in producing various kinds of diagrams supported by the tool, including most of the SysML diagrams and attack-defense graphs, or AMULET mutation scripts for supporting incremental MDE processes.”
Exploring Gen Z Perspectives Toward ChatGPT in Academic Contexts
Venugopal, K. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-4268-8.ch004 · Cited by 12
Unmoderated Usability Studies Evolved: Can GPT Ask Useful Follow-up Questions?
Kuric, E., Demcak, P., & Krajcovic, M. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2427978 · Cited by 21
“The ability of GPT-based large language models (LLMs) to generate replies that satisfy human users renders them a potent catalyst for innovation in the field of human-computer interaction (Ahmad et al., 2023; Tabone & de Winter, 2023; Ye et al., 2023).”
Autonomous Agents in Software Development: A Vision Paper
Rasheed, Z., Waseem, M., Sami, M., Kemell, K., Ahmad, A., Nguyen‐Duc, A., Systä, K., & Abrahamsson, P. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-72781-8_2 · Cited by 53
“Several studies (e.g., [1], [4],[32]) has explored GPT models in SE.”
Self-citationA Student-Centric Evaluation Survey to Explore the Impact of LLMs on UML Modeling
Al-Ahmad, B. I., Alsobeh, A., Meqdadi, O., Shaikh, N., & Kabir, M. F. (2025) · Information · doi:10.3390/info16070565 · Cited by 28
Artificial Intelligence-Based Tools in Software Development Processes: Application of ChatGPT
Özpolat, Z., Yıldırım, Ö., & Karabatak, M. (2023) · European Journal of Technic · doi:10.36222/ejt.1330631 · Cited by 33
“Ahmad et al. [6] presented research on how architecture-centric software engineering (ACSE) professionals can benefit from ChatGPT.”
BackgroundTechnologies for Children's AI Learning: Design Features and Future Opportunities
Jia, K., & Yu, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713443 · Cited by 17
“This lack of comprehensive understanding of current AI learning tools and their design can lead to missed opportunities for fostering AI literacy among young people, which is increasingly critical due to the growing prevalence of AI applications (e.g., ChatGPT [3]) and their societal impacts.”
Generating SysML Behavior Models via Large Language Models: an Empirical Study
Wang, Y., Ge, N., Liu, J., Cao, Z., Chen, Z., & Hu, C. (2025) · Asia-Pacific Symposium on Internetware · doi:10.1145/3755881.3755926 · Cited by 22
Nikolaidis, N., Flamos, K., Gulati, K., Feitosa, D., Ampatzoglou, A., & Chatzigeorgiou, A. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C) · doi:10.1109/saner-c62648.2024.00018 · Cited by 23
“According to Ahmad et al. [11] ChatGPT could be used to help in the architecture stage of software design, but more studies are required.”
MethodAutoTestGPT: A System for the Automated Generation of Software Test Cases based on ChatGPT
Liu, H., Liu, L., Yue, C., Wang, Y., & Deng, B. (2024) · Journal of Software · doi:10.17706/jsw.19.4.108-120 · Cited by 13
On the Utility of Domain Modeling Assistance with Large Language Models
Chaaben, M. B., Burgueño, L., Dávid, I., & Sahraoui, H. A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3744920 · Cited by 32
“It has been used in numerous tasks in the overall software development lifecycle, from requirements engineering [45] through software architecture [2] to testing [23]; as well as organizing software engineering knowledge by accelerating literature surveys [52, 53].”
Benchmarking AI design skills: insights from ChatGPT's participation in a prototyping hackathon
Ege, D. N., Øvrebø, H. H., Stubberud, V., Berg, M. F., Steinert, M., & Vestad, H. (2024) · Proceedings of the Design Society · doi:10.1017/pds.2024.202 · Cited by 17
“It has already started transforming the way engineers write code (Ahmad et al., 2023), and it has been used as a tool for optimising 3D printing processes (Badini et al., 2023) and to generate functional 3D models iteratively (Nelson et al., 2023).”
Analyzing Developer-ChatGPT Conversations for Software Refactoring: An Exploratory Study
Chavan, O. S., Hinge, D. D., Deo, S. S., Wang, Y. (., & Mkaouer, M. W. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3645082 · Cited by 25
“The inception of OpenAI’s ChatGPT in 2022 showcased multiple break-throughs in various software engineering applications, in general, [2, 7–10, 13, 15], and refactoring, in particular, [1, 6, 11].”
MethodLLM-Based Automation of COSMIC Functional Size Measurement From Use Cases
Vito, G. D., Martino, S. D., Ferrucci, F., Gravino, C., & Palomba, F. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3554562 · Cited by 13
“By providing a suitable prompt, LLMs can generate accurate responses [58], [59].”
Enabling Architecture Traceability by LLM-based Architecture Component Name Extraction
Fuchß, D., Liu, H., Hey, T., Keim, J., & Koziolek, A. (2025) · IEEE International Conference on Software Architecture (ICSA) · doi:10.1109/icsa65012.2025.00011 · Cited by 30
“Besides, LLMs have been used for various architecture and modeling tasks [20], [21], including supporting design decision making [22], [23], modeling tasks [24], [25], and software architecture analysis and generation [26], [27].”
ChatGPT as a Fullstack Web Developer - Early Results
Abrahamsson, P., Anttila, T., Hakala, J., Ketola, J., Knappe, A., Lahtinen, D., Liukko, V., Poranen, T., Ritala, T.-M., & Setälä, M. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-48550-3_20 · Cited by 10
Pooryousef, V., Cordeil, M., Besançon, L., Bassed, R., & Dwyer, T. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456212 · Cited by 16
“Many studies have started to evaluate capabilities of LLMs in different applications, such as education [68, 77], medical diagnosis [13,41,60], software development [2,66], and business [53].”
BackgroundZahra, S., Samra, M., & Gizawi, L. E. (2025) · Buildings · doi:10.3390/buildings15030356 · Cited by 9
A Data Science Course Utilizing GenAI
Browning, J. W., Bustard, J. D., Anderson, N., & Galway, L. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893452 · Cited by 13
“Ahmad et al. [7] expand on this, observing that ChatGPT can generate diverse software artifacts but still requires human oversight for consistency and ethical considerations in software architecture.”
Generating Java code pairing with ChatGPT
Zhao, Z., Zhang, N., Yu, B., & Duan, Z. (2024) · Theoretical Computer Science · doi:10.1016/j.tcs.2024.114879 · Cited by 11
Kim, D., & Ming, H. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107787 · Cited by 13
Tinnes, C., Welter, A., & Apel, S. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00112 · Cited by 20
Kruchten, P. B., & Gregory, P. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-48550-3 · Cited by 12
Cautious Optimism: The Influence of Generative AI Tools in Software Development Projects
Mbizo, T., Oosterwyk, G., Tsibolane, P., & Kautondokwa, P. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64881-6_21 · Cited by 20
Mining software repositories for software architecture — A systematic mapping study
Soliman, M., Albonico, M., Malavolta, I., & Wortmann, A. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107677 · Cited by 18
Comparative analysis of design pattern implementation validity in LLM-based code refactoring
Kim, D. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112519 · Cited by 11
Exploring GenAI in Software Development: Insights from a Case Study in a Large Brazilian Company
Pereira, G. V., Jackson, V., Prikladnicki, R., van der Hoek, A., Fortes, L., Araújo, C., Coelho, A., Chelli, L., & Ramos, D. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00035 · Cited by 24
“Additionally, several recent studies have explored the use of GenAI in various software development activities, e.g., generating architecture [5], source code generation [6], and program repair [7].”
LLM as a Code Generator in Agile Model Driven Development
Sadik, A. R., Brulin, S., Olhofer, M., Ceravola, A., & Joublin, F. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-96841-9_10 · Cited by 7
Generative AI for Code Generation: Software Reuse Implications
Kapitsaki, G. M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66459-5_3 · Cited by 14
Applying large language models to issue classification: Revisiting with extended data and new models
Aracena, G., Luster, K., Santos, F., Steinmacher, I., & Gerosa, M. A. (2025) · Science of Computer Programming · doi:10.1016/j.scico.2025.103333 · Cited by 11
“Beyond issue classification, Liu et al. [26] employed a pre-trained transformer-based model for code understanding and generation, while Ahmad et al. [2] explored service-based software us-ing AI-powered bots.”
Süße, T., Kobert, M., Grapenthin, S., & Voigt, B.-F. (2023) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-42622-3_49 · Cited by 8
Characterizing Developers' Behaviors in LLM -Supported Software Development
Wang, W., Ning, H., Qian, S., Zhang, G., & Wang, Y. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00156 · Cited by 9
“Ahmad et al. [20] explored the potential of LLMs, specifically DevBots like ChatGPT, in architecting complex software-intensive systems, demonstrating their utility in human-bot collaborative processes.”
BackgroundThe Transformative Influence of LLMs on Software Development & Developer Productivity
Jalil, S. (2025) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa65407.2025.11166110 · Cited by 22
“2) Privacy: Within industry spheres, developers often express concerns about the privacy used by AI Pair Programmers [23], [29], [34], [37].”
InfluentialOn the Generalizability of Deep Learning-based Code Completion Across Programming Language Versions
Ciniselli, M., Martin-Lopez, A., & Bavota, G. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644411 · Cited by 9
From Problem Descriptions to User Stories: Utilizing Large Language Models through Prompt Chaining
Sharma, A., Chaturvedi, A., & Tripathi, A. K. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10724709 · Cited by 7
“Through a Human-DevBot partnership, Ahmed et al. [17] investigated the potential of ChatGPT in Architecture-centric Software Engineering (ACSE).”
Naqbi, H. A., Bahroun, Z., & Ahmed, V. (2025) · Sustainability · doi:10.3390/su17219914 · Cited by 10
“Evolving from conceptual frameworks to powerful real-world applications, generative AI is revolutionizing industries by enabling unprecedented levels of creativity, efficiency, and problem-solving capabilities [1].”
AI-Enhanced Leadership Decision Support System Using ChatGPT for Ethical Learning
Balamurugan, M., Raghu, N., Dayananda, L. N., Sahoo, S. K., Shankaregowda, S. A., Kannanugo, N., Deepthi, M., & Vinod Kumar, M. (2024) · Advances in business strategy and competitive advantage book series · doi:10.4018/979-8-3693-1946-8.ch008 · Cited by 7
ChatReliability: A Customized CoPilot Based on LLM to Facilitate Design for Reliability
Tadaka, K. B., Cassandro, R., Zhang, W., & Li, Z. S. (2024) · IEEE Technology & Engineering Management Conference - Asia Pacific (TEMSCON-ASPAC) · doi:10.1109/temscon-aspac62480.2024.11024829 · Cited by 6
“Recently, more sophisticated architectures have been created because of the adoption of AI and NLP [33].”
A Framework for Patient-Centric Medical Image Management using Blockchain Technology
Aljaloud, A. S. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140883 · Cited by 6
“This fragmentation often results in cumbersome data management and compromises overall patient safety, as health information systems within hospitals, medical clinics, laboratories, and pharmacies remain separate entities that are prone to data security threats or compromised health-critical information [5].”
BackgroundA Report on the Use of ChatGPT in Software Engineering and Systems Analysis Courses
Albonico, M., & Varela, P. J. (2023) · Brazilian Symposium on Software Engineering · doi:10.1145/3613372.3614189 · Cited by 7
Redefining the entrepreneurial discovery process for smarter specialization in Europe
Ganzaroli, A. (2024) · European Planning Studies · doi:10.1080/09654313.2024.2324041 · Cited by 5
“Additionally, software like ChatGPT enables moderators to assess levels of agreement and disagreement in real-time debates, supporting e ff ective collaboration between stakeholders (Ahmad et al. 2023).”
ChatGPT as a Full-Stack Web Developer
Liukko, V., Knappe, A., Anttila, T., Hakala, J., Ketola, J., Lahtinen, D., Poranen, T., Ritala, T.-M., Setälä, M., Hämäläinen, H., et al. (2024) · Springer eBooks · doi:10.1007/978-3-031-55642-5_9 · Cited by 5
Towards AI for Software Systems
Eskandani, N., & Salvaneschi, G. (2024) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1145/3664646.3664767 · Cited by 7
Katz, K., Moshtari, S., Mujhid, I., Mirakhorli, M. T., & Garcia, D. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00012 · Cited by 8
Who’s Who? LLM-assisted Software Traceability with Architecture Entity Recognition
Fuchß, D., Liu, H., Corallo, S., Hey, T., Keim, J., von Geisau, J., & Koziolek, A. (2026) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3807453 · Cited by 9
“Among those techniques, methods like Vector Space Model (VSM) and Latent Semantic Indexing (LSI) are commonly used [3].”
Dishop, C. R., Brown, A. S., Chao, P.-Y., Kuznetsov, A. V., & Woolley, A. W. (2025) · Journal of Business and Psychology · doi:10.1007/s10869-025-10068-x · Cited by 8
Metadata
- Authors
- Ahmad, A., Waseem, M., Liang, P., Fahmideh, M., Aktar, M. S., & Mikkonen, T.
- Affiliations
- Lancaster University (GB)
- University of Jyväskylä (FI)
- Wuhan University (CN)
- University of Southern Queensland (AU)
- Venue
- Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering
- Publisher
- ACM
- Published
- 2023-06-14
- Pages
- 279-285
- OpenAlex
- W4380568688
- Cited by
- 168
- References
- 12
- Citations by year
- 2026: 21 · 2025: 55 · 2024: 61 · 2023: 31
- FWCI
- 29.82
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Conference on Evaluation and Assessment in Software Engineering (EASE) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Jyväskylä University Digital Archive (University of Jyväskylä) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China (62172311)
- Jyväskylän Yliopisto
- Topics
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- software architecture, ChatGPT, large language models, AI decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.