How does Machine Learning Change Software Development Practices?
The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Adding an ability for a system to learn inherently adds uncertainty into the system. Given the rising popularity of incorporating machine learning into systems, we wondered how the addition alters software development practices. We performed a mixture of qualitative and quantitative studies with 14 interviewees and 342 survey respondents from 26 countries across four continents to elicit significant differences between the development of machine learning systems and the development of non-machine-learning systems. Our study uncovers significant differences in various aspects of software engineering (e.g., requirements, design, testing, and process) and work characteristics (e.g., skill variety, problem solving and task identity). Based on our findings, we highlight future research directions and provide recommendations for practitioners.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is complex for practitioners to comprehend the scopes and impacts of architectural changes [95, 103].”
From Software Architecture in Practice: Challenges and Opportunities (2023) · cites this paper as [103]
In the citing paper
“Moreover, software engineering has had decades of development in data management, which could be readily adapted to AI practices in many cases [190].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [190]
In the citing paper
“Exploratory processes in ML share many similarities to scientific programming, and would benefit from lessons learned in embracing uncertainty [190].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [190]
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 1×
“We conducted a thematic analysis [8] to process the recorded interviews by following the steps below: Transcribing and Coding.”
MethodA Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 1×
“We then use the Cohen’s Kappa measure [10] to examine the agreement between the two labelers.”
MethodHow Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808 · cited 1×
“[15], conducting 12 to 15 interviews of a homogeneous group is adequate to reach saturation.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 2×
“Machine learning (ML) has progressed dramatically over the past three decades, from a laboratory curiosity to a practical technology in widespread commercial use [19].”
BackgroundHumphrey, S. E., Nahrgang, J. D., & Morgeson, F. P. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.5.1332 · Cited by 2,303 · cited 1×
“[18] in applied psychology, which consists of 21 work characteristics, e.”
BackgroundProbabilistic machine learning and artificial intelligence
Ghahramani, Z. (2015) · Nature · doi:10.1038/nature14541 · Cited by 1,995 · cited 1×
“Uncertainty plays a fundamental role in probabilistic modeling [14]: Observed data can be consistent with various models, and thus which model is appropriate given the data is uncertain.”
BackgroundDeepXplore: Automated Whitebox Testing of Deep Learning Systems
Pei, K., Cao, Y., Yang, J., & Jana, S. (2017) · Proceedings of the 26th Symposium on Operating Systems Principles · doi:10.1145/3132747.3132785 · Cited by 1,217 · cited 1×
“To tackle the ML specific issues, recent studies have put effort into building tools for testing [26], [30], [36], [39] and debugging [16], [27], [28] of machine learning code, and creating frameworks and environments to support development of ML systems [3], [6].”
DeepTest: automated testing of deep-neural-network-driven autonomous cars
Tian, Y., Pei, K., Jana, S., & Ray, B. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180220 · Cited by 1,217 · cited 1×
“To tackle the ML specific issues, recent studies have put effort into building tools for testing [26], [30], [36], [39] and debugging [16], [27], [28] of machine learning code, and creating frameworks and environments to support development of ML systems [3], [6].”
MethodSoftware Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135 · cited 3×
“Most similar to our study is an empirical investigation of integrating AI capabilities into software and services and best practices from Microsoft teams [4].”
BackgroundMethodInfluential
Research works printed but not linked: 3
- romano, 2006, Appropriate statistics for ordinal level data: Should we really be using t-test and cohen'sd for evaluating group differences on the nsse and other surveys, Proc Annu Meet Florida Assoc Institutional Res, 1
- 10.1007/bf02729815 · doi:10.1007/bf02729815
- ng, 2017, Machine Learning Yearning
Cited by
Items 101–200 of 201
Alshammari, F. H. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/1960684 · Cited by 15
“[13] were curious about the impact of machine learning on software development techniques, given the growing popularity of this approach.”
BackgroundTraini, L., & Cortellessa, V. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3266041 · Cited by 14
Unraveling Code Clone Dynamics in Deep Learning Frameworks
Assi, M., Hassan, S., & Zou, Y. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3721125 · Cited by 13
“We collect 37 metrics as detailed in Section 2.5.”
InfluentialNecula, S.-C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.13229 · Cited by 13
Sklavenitis, D., & Kalles, D. (2025) · Applied Sciences · doi:10.3390/app15137165 · Cited by 15
How to Support ML End-User Programmers through a Conversational Agent
Garcia, E. J. A., Pimentel, J. F., Feng, Z., Gerosa, M. A., Steinmacher, I., & Sarma, A. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608130 · Cited by 19
Lin, C.-T., & Huang, S.-J. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107449 · Cited by 13
Brea, E. (2024) · Research Policy · doi:10.1016/j.respol.2024.105008 · Cited by 10
An Exploratory Study of V-Model in Building ML-Enabled Software: A Systems Engineering Perspective
Wu, J. J. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644951 · Cited by 13
State and tendency: an empirical study of deep learning question&answer topics on Stack Overflow
Zhao, H., Li, Y., Liu, F., Xie, X., & Lin, C. (2021) · Science China Information Sciences · doi:10.1007/s11432-019-3018-6 · Cited by 13
A graph-theoretic framework for quantitative analysis of angiogenic networks
Okoro, G., Wityk, P., Nelappana, M. B., Jackiewicz, K. A., Kucharczyk, V. Z., Tigranyan, A., Applegate, C. C., Dobrucki, I. T., & Dobrucki, L. W. (2025) · BioData Mining · doi:10.1186/s13040-025-00478-1 · Cited by 10
Alatawi, M. N. (2024) · Iran Journal of Computer Science · doi:10.1007/s42044-024-00194-9 · Cited by 11
“This study [23] reveals differences in software engineering (including requirements, design, testing, and process) and workplace characteristics.”
An Exploratory Study on Machine Learning Model Management
Latendresse, J., Abedu, S., Abdellatif, A., & Shihab, E. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3688841 · Cited by 14
Understanding Newcomers’ Onboarding Process in Deep Learning Projects
Han, J., Zhang, J., Lo, D., Xia, X., Deng, S., & Wu, M. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3353297 · Cited by 9
“We then applied open card sorting [23], [59], [64] to analyze these answers following the next steps: 1) the first two authors separately read these answers, 2) they generated the initial codes for each answer separately, 3) searching if the cards have already existed, 4) sorting the generated cards…”
BackgroundMethodInfluentialSelf-citationShankar, S. P., & Chaudhari, S. S. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i6s.6944 · Cited by 8
A framework for Requirements specification of machine-learning systems
Wang, X. (2022) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2022-143 · Cited by 8
Constructing Dependable Data-Driven Software With Machine Learning
Pahl, C., & Azimi, S. (2021) · IEEE Software · doi:10.1109/ms.2021.3067940 · Cited by 7
Shifting Left for Early Detection of Machine-Learning Bugs
Liblit, B., Luo, L., Molina, A., Mukherjee, R., Patterson, Z., Piskachev, G., Schäf, M., Tripp, O., & Visser, W. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-27481-7_33 · Cited by 7
“For a given programming language, notebook developers invest more time into notebooks between published revisions than traditional developers [11, 35].”
BackgroundReturn on Investment in Machine Learning: Crossing the Chasm between Academia and Business
Mizgajski, J. T., Szymczak, A., Morzy, M., Augustyniak, Ł., Szymański, P., & Żelasko, P. (2020) · Foundations of Computing and Decision Sciences · doi:10.2478/fcds-2020-0015 · Cited by 7
“A recent study [110] presents the results of a large survey of software engineers on their experiences when developing ML-based products.”
BackgroundExtracting corrective actions from code repositories
Bugayenko, Y., Daniakin, K., Farina, M., Jolha, F., Kruglov, A. V., Succi, G., & Witold Pedrycz, C. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528517 · Cited by 7
“This instructive failure prompted us to speculate that it might be possible to relate all other anomalies and associated preventive and corrective actions by using Machine Learning (ML) techniques [4, 5] with the intent of producing an advisory system, deployable during all phases of software development, capable of e…”
BackgroundKim, D. H., Jeong, D., Yadgarova, S., Shin, H., Son, J., Subramonyam, H., & Kim, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714044 · Cited by 8
Liang, A., Murphy-Hill, E. R., Weimer, W. R., & Huang, Y. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3437355 · Cited by 9
“On one hand, corrections to the false discovery rate (FDR) are common in the software engineering literature (e.g. [96], [97], [98]).”
MethodHeo, J., & Lee, U. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610189 · Cited by 7
Rodrigues, G. A. P., Fernandes, P. A. G., Serrano, A. L. M., Filho, G. P. R., Vergara, G. F., Bispo, G. D., Albuquerque, R. D. O., & Gonçalves, V. P. (2025) · Journal of Internet Services and Applications · doi:10.5753/jisa.2025.5034 · Cited by 6
Gracy, J. A., Parthasarathy, S., & Sivagurunathan, S. (2024) · International Journal of Computing and Digital Systems · doi:10.12785/ijcds/160130 · Cited by 8
“By considering and implementing these factors, software startup companies can enhance their software project performance and increase their chances of delivering successful products to the market. https://journal.uob.edu.bh [35],[36],[21] [25],[22] Metrics & Measurement”
A Survey on Automated Code Evaluation Systems and Their Resources for Code Analysis
Rahman, M. M., Watanobe, Y., & Hamada, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36822-6_33 · Cited by 5
Component-based Approach to Software Engineering of Machine Learning-enabled Systems
Indykov, V. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644976 · Cited by 6
Ferreira, A. L., & Fernandes, J. M. (2024) · IEEE Software · doi:10.1109/ms.2024.3415750 · Cited by 8
Iftikhar, A., Musa, S., Alam, M. M., Ahmed, R., Su’ud, M. M., Khan, L. M., & Ali, S. M. (2022) · Journal of Information Technology Research · doi:10.4018/jitr.299385 · Cited by 5
“In several cases, machine learning capacities are applied to a program, like ML and ML applications software systems, methods and libraries that provide ML features (Wan et al., 2019).”
MethodÖzçevik, Y., & Altay, O. (2023) · SoftwareX · doi:10.1016/j.softx.2023.101499 · Cited by 5
Mohapatra, H., Pramanik, S., & Mishra, S. R. (2025) · Artificial intelligence-enhanced software and systems engineering · doi:10.1007/978-3-031-88188-6_3 · Cited by 4
Leça, M. D. M., & Santos, R. D. S. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00021 · Cited by 10
“This evolution requires developers to not only adopt new approaches but also expand their skill sets [12].”
Towards Requirements Engineering for RAG Systems
Sporsem, T., & Ulfsnes, R. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3757040 · Cited by 5
“There are already studies describing how software engineering (see e.g. Kalinowski et al. [10], Wan et al. [28]) and requirements engineering (see e.g. Amershi et al. [2], Kim et al. [11]) change when developing AI systems.”
A Preliminary Framework for Intersectionality in ML Pipelines
Turcios, M. N., Boyd, A. E., Smith, A. D. R., & Johnson, B. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728692 · Cited by 4
Automatically debugging AutoML pipelines using maro: ML automated remediation oracle
Dolby, J., Tsay, J., & Hirzel, M. J. (2022) · doi:10.1145/3520312.3534868 · Cited by 5
“Debugging in particular is challenging due to errors hiding in data rather than code [16, 30] and the sheer amount of effort involved and the impossibility of manual evaluation due to the potentially millions of parameters to inspect [3, 15, 18].”
BackgroundVorgehensmodelle in der Softwareentwicklung
Broy, M., & Kuhrmann, M. (2021) · Xpert.press · doi:10.1007/978-3-662-50263-1_3 · Cited by 4
Quaranta, L. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510454.3517055 · Cited by 6
“3517055 Such an integration, however, poses a large number of complex challenges [2, 8, 13–15], in part inherited from the fields of Software Engineering (SE) and Data Science (DS), in part completely new [31], arising from the need to translate lab model prototypes into production-ready AI/ML components [10–12].”
BackgroundImplementing Federated Learning based on RainForest Model
Karim, M., Kundu, N. K., Saha, D., Kabir, S., Mim, S. A., & Farid, D. M. (2023) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct57861.2023.10126333 · Cited by 4
“Additionally, there is a possibility that client data will be leaked or abused by an intruder or the developer himself [3].”
BackgroundFactors influencing Stress, Anxiety, and Depression in Students regarding The Thesis Process
Nguyen, H. A., Duong, B. C., Le, T. D., Le, C. A., Huynh, T. Q., Tran, V. N., Nguyen, P. T., & Nguyen, T. T. H. (2025) · JOURNAL OF MENTAL HEALTH AND CLINICAL PSYCHOLOGY · doi:10.29245/2578-2959/2025/1.1339 · Cited by 4
Task Decomposition and RAG as Design Patterns for LLM-Based Systems
Ayala, O. M. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00049 · Cited by 3
“Integrating AI models into software already adds a new set of complex design choices to software engineering [1], yet this complexity has increased due to an LLM’s expanding abilities to generate almost any kind of output.”
Ferreira, R. C. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00041 · Cited by 4
“…which become visible in the software architecture: • they are harder to modularize, since data modeling and data processing are highly coupled [11]; • they require an understanding of ML principles and techniques, with implications to the number of ML models required, their size, versioning,…”
A checklist of quality concerns for architecting ML-intensive systems
Bucaioni, A., Kazman, R., & Pelliccione, P. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112612 · Cited by 4
Software Engineering in Machine Learning Applications
Vayadande, K. B., Munde, K., Bhosle, A. A., Sawant, A. R., & Kulkarni, A. M. (2024) · Wiley eBooks · doi:10.1002/9781394214167.ch12 · Cited by 3
Abualsaud, R. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00024 · Cited by 3
“Its behavior is influenced by the input dataset, hyperparameter tuning, and the choice of a learning algorithm, all of which collectively determine the final model [25], [26].”
On the sustainability of deep learning projects: Maintainers' perspective
Han, J., Liu, J., Lo, D., Zhi, C., Chen, Y., & Deng, S. (2023) · Journal of Software Evolution and Process · doi:10.1002/smr.2645 · Cited by 3
Self-citationNahar, N., Zhou, S., Lewis, G. A., & Kästner, C. (2021) · arXiv (Cornell University) · Cited by 6
IDENTIFYING CONCERNS WHEN SPECIFYING MACHINE LEARNING-ENABLED SYSTEMS: A PERSPECTIVE-BASED APPROACH
Villamizar, H. (2023) · doi:10.17771/pucrio.acad.65972 · Cited by 3
Data-Dependent Goal Modeling for ML-Enabled Law Enforcement SystemsPreprint
Alrajeh, D., Nowack, V., Benjamin, P., Thomas, K., Hobson, W., Muñoz, C. G., Hamilton-Giachritsis, C. E., Kloess, J. A., Woodhams, J., Butler, D., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.06237 · Cited by 5
CoCo Challenges in ML Engineering Teams: How to Collaboratively Build ML-Enabled Systems
Azamnouri, A. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00036 · Cited by 3
“It is also essential when changes happen throughout the project since the requirements for ML-enabled systems change continuously [23].”
Automatic Identification of Machine Learning-Specific Code Smells
Hamfelt, P., Britto, R., Rocha, L. S., & Almendra, C. (2025) · Simpósio Brasileiro de Engenharia de Software (SBES) · doi:10.5753/sbes.2025.9949 · Cited by 2
The Many Shades of Clouds: How Law Fails (Us) in Seeing Power in the Digital Economy
Terzis, P., Mittal, A., & van Hoboken, J. V. J. (2026) · Regulation & Governance · doi:10.1111/rego.70149 · Cited by 3
LLMs for industrial databases: an agro-food production plant use case
Dranca, L., Donate, P., Sanguesa, J. A., Garrido, P., Torres‐Sanz, V., & Martínez, F. J. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1764367 · Cited by 2
“As noted by Wan et al. (2019), the development of ML-based systems introduces additional uncertainty in both requirements and quality, and demands greater effort in the creation of test sets that are specific and representative of real use cases.”
A Case Study on Data Science Processes in an Academia-Industry Collaboration
Silva, S., & de França, B. B. N. (2023) · Brazilian Symposium on Software Quality (SBQS) · doi:10.1145/3629479.3629514 · Cited by 3
Integrating Optimization Techniques in Software Development Life Cycle - SLR
Ali, R., & Bahrami, M. R. (2024) · International Conference on Electrical, Communication, and Computer Engineering (ICECCE) · doi:10.1109/icecce63537.2024.10823491 · Cited by 2
Ginde, G., Ruhe, G., & Saunders, C. (2025) · IEEE Access · doi:10.1109/access.2025.3556313 · Cited by 2
“Rather it needs to be updated and improved continuously over a period either by feeding new data into the system or by retraining new versions of the system [57].”
Conceptual Mappings of Conventional Software and Machine Learning-based Applications Development
Angel, S., & Namin, A. S. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00193 · Cited by 2
Development and Deployment Challenges of Machine Learning Systems
Honkanen, T., O’Dwyer, J., Salminen, V., & Jussila, J. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64359-0_15 · Cited by 2
Technical Debt in the Peer-Review Documentation of R Packages: a rOpenSci Case Study
Codabux, Z., Vidoni, M. C., & Fard, F. H. (2021) · Figshare · doi:10.5281/zenodo.4589572 · Cited by 2
Danter, W. R. (2024) · medRxiv · doi:10.1101/2024.08.21.24312358 · Cited by 2
“This indicates that the GOF HNRNPH2 mutations are associated with significant impairment of ability to perform ADL that begins at birth and continues across all age cohorts studied.”
BackgroundUnderstanding Developer Information Needs for Software with Embedded Natural Language PromptsPreprint
Liang, J. T., Yang, C., Sergeyuk, A., Breaux, T. D., & Myers, B. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.17264 · Cited by 2
“More support for assessing dataset quality to avoid training-serving skew [50, 63] (i.e., when a model’s training data does not generalize to production).”
Libertz, D., Belmihoub, K., Schreiber, C., & Blankenship, L. (2024) · Computers & composition/Computers and composition · doi:10.1016/j.compcom.2024.102904 · Cited by 2
From code to security: machine learning approaches in android vulnerability detection
Arikan, K. E., Dogan, S. M., Yılmaz, E. N., & Gönen, S. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-01190-1 · Cited by 2
Schott, K., Papenmeier, A., Hienert, D., & Kern, D. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787872 · Cited by 1
To Build or Not to Build? Factors that Lead to Non-Development or Abandonment of AI Systems
Chappidi, S., & Singh, J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806736 · Cited by 2
“Model training and inference can also be resource intensive lifecycle stages [170], influencing decision-making where resources such as GPUs or data center storage are limited.”
Data-Dependent Goal modeling for ML-Enabled Law Enforcement Systems
Alrajeh, D., Nowack, V., Benjamin, P., Thomas, K., Hobson, W., Muñoz, C. G., Hamilton-Giachritsis, C. E., Kloess, J. A., Woodhams, J., Butler, D., et al. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786581.3786944 · Cited by 1
Dhiman, H., Michael Fellmann, M. A., & Röcker, C. (2022) · Lecture notes in business information processing · doi:10.1007/978-3-031-16947-2_7 · Cited by 1
Ghosh, S., Mitra, B., & De, P. (2022) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii55700.2022.9953819 · Cited by 1
“The frequency of retraining can be decided based on the data volume or when the distribution of stream data changes significantly than training data to overcome the retraining overhead [43], [44].”
BackgroundIntuitionistic Fuzzy Set Based Ensemble Approaches for Software Bug Triaging
Panda, R. R., & Nagwani, N. K. (2023) · OITS International Conference on Information Technology (OCIT) · doi:10.1109/ocit59427.2023.10430843 · Cited by 1
A Requirements Engineering Process for Machine Learning Innovation Projects
Martins, M. C., Kudo, T. N., & Bulcão-Neto, R. F. (2024) · Workshop em Engenharia de Requisitos (WER) · doi:10.29327/1407529.27-24 · Cited by 2
“This paper proposes an RE process for ML-enabled systems in innovation projects that would guide requirements analysts in understanding the data and identifying characteristics and their distribution, which is essential to achieve a high-quality ML model [27].”
MethodContinuous Integration for Machine Learning Experiments Reproducibility: a Practical Study
Andrade, A. M., Pereira, M. B., Silveira, S., Linhares, F. I. F., Neto, A., Andrade, R., & Araújo, Í. (2021) · International Symposium on Information Science and Engineering · doi:10.5753/ise.2021.17279
“[6] presented a review of software engineering methods applied in MLmodel development.”
BackgroundMethodInfluential2021 Index IEEE Transactions on Software Engineering Vol. 47
(2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135674
Determining the Course Interests of Gifted Students using Machine Learning
Arapoglu, O., & Ciftci, R. T. (2022) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec56263.2022.9998191
“Wan et al. reveal the important differences in various parts of software engineering and the business characteristics between the development of machine learning systems and the development of non-machine learning systems [20].”
BackgroundScalable Neuroevolution of Ensemble Learners
Merten, M., Krauss, R., & Drechsler, R. (2023) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3583133.3590711
“Due to the increased complexity, interest in machine learning has been growing in recent decades in numerous areas such as the video game industry and medicine [11], making it an essential part of the software design process for many applications [17].”
BackgroundGarbage in, Garbage Out: A Case Study on Defective Product Prediction in Manufacturing
Çolhak, F., Uçar, B. E., Saygut, İ., Düzgün, B., Demirkıran, F., & Dağ, H. (2023) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk59864.2023.10286707
Methods for Requirements Engineering, Verification, Security, Safety, and Robustness in AIoT Systems
Pasin, M., Ménétrey, J., Felber, P. A., Schiavoni, V., Heyn, H.-M., Knauss, E., Khurshid, A., & Raza, S. (2023) · River Publishers eBooks · doi:10.1201/9781032632407-12
Transcompiler CNN-GMN Based Cross Language Clone Detection and Algorithm Recognition
Singh, H., Bhutiya, T., Goyal, A., Rachana, K., & Diyora, U. S. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.57722
Sabir, R., Hassan, S., Ittifaq, M. H., Iqbal, M. W., Raza, M., Raza, A., & Fatima, P. (2024) · Bulletin of Business and Economics (BBE) · doi:10.61506/01.00252
“In addition to the difficulties faced by traditional software systems, these issues require new approaches to architecture for ML-based systems (Wan et al., 2019).”
BackgroundHarp, A. J., & Maczko, K. A. (2024) · Die Bodenkultur Journal of Land Management Food and Environment · doi:10.2478/boku-2024-0002
ANALISIS PENGGUNAAN KOMBINASI WORD EMBEDDING, RESNET, DAN GRU PADA MODEL PIX2CODE
Rahman, F. T., Abdillah, F. R., Semiawan, T., & Syakrani, N. (2024) · JTT (Jurnal Teknologi Terapan) · doi:10.31884/jtt.v10i1.356
Knowledge Overlap and Iterative Development in ML Projects: An Information Processing View
Krancher, O., Oshri, I., & Kotlarsky, J. (2024) · IT University Of Copenhagen (IT University of Copenhagen)
Haddag, A., Guerrero, E., Yedjour, H., & Guerrero-Lebrero, M. P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02728-3_20
Nguyen, H. A., Le, N. V., Le, N. T., Do, T. T. H., Dao, N. B., Nguyen, T. N., Duong, D. M. T., Tran, D. H. N., Huynh, T. Q., Nguyen, H.-L., et al. (2025) · Journal of Ethnopharmacology · doi:10.1016/j.jep.2025.120864 · Cited by 1
DIP-AI: A Discovery Framework for AI Innovation Projects
Martins, M. C., Rocha, L. E. C., Romão, L., Kudo, T. N., Kalinowski, M., & Bulcão-Neto, R. F. (2025) · Simpósio Brasileiro de Qualidade de Software (SBQS) · doi:10.5753/sbqs.2025.13834 · Cited by 1
Formal Methodologies And Supporting Tools Towards Autonomous Software Generation
Xu, Y. (2022) · Libraries and Cultural Resources (University of Calgary) · doi:10.11575/prism/dspace/41326
Watteel, M. (2024) · Document Server@UHasselt (UHasselt)
Koneoppimisen merkitys ohjelmistotestaukselle organisaatioissa
Annamaa, S. (2023) · Jyväskylä University Digital Archive (University of Jyväskylä)
Abordagens ao desenvolvimento de software usando ferramentas de inteligência artificial
Brito, M. T. D. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Lorenz, P., Steinbeck, F., Fricke, F., Mai, F., Bergmann‐Ewert, W., Wossidlo, C., Reisinger, E. C., & Müller‐Hilke, B. (2026) · Scandinavian Journal of Immunology · doi:10.1111/sji.70088
AI Inclusion in Software Engineering Education and Employability: A Decisive Investigation
Mishra, K. N., Mishra, A., Mishra, D., & Barwal, P. N. (2025) · International Conference on Computational Intelligence and Knowledge Economy (ICCIKE) · doi:10.1109/iccike67021.2025.11318202
Pramono, M. A., & Sholiq, S. (2026) · International journal of intelligent engineering and systems · doi:10.22266/ijies2026.0430.27
Lignin paradox: selective property enhancement in particleboard
Martinka, J., Lykidis, C., Wallisch, V., Rantuch, P., Štefko, T., Wachter, I., & Mantanis, G. I. (2026) · European Journal of Wood and Wood Products · doi:10.1007/s00107-026-02476-4
“|) calculated in MATLAB 2025b. Values of |δ| ≥ 0.474 were interpreted as a large effect size, following Romano and Kromrey (2006) and Wan et al. (2020).”
InfluentialExploring CoCo Challenges in ML Engineering Teams: Insights From the Semiconductor IndustryPreprint
Azamnouri, A., Haug, M., Woltmann, L., Fritz, M., Bogner, J., & Wagner, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.07389
“…in software engineering for ML systems and machine learning operations (MLOps) in general (Amershi et al., 2019; Lima et al., 2022; Honkanen et al., 2022; Haug et al., 2025), and highlight the complexities of integrating ML into software systems (Wan et al., 2020; Kalinowski et al., 2025).”
Huang, J. Y. H., Jiang, J. J., & Wu, J. (2026) · Journal of the Association for Information Systems
Security Built-In, not Bolted-On: A Structured Taxonomy for Early Secure Software Development
Goel, S., & Mazzara, M. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-23347-9_1
Mỹ, N. X., Toàn, N. V., Anh, T. Q., & Hải, V. M. (2026) · Tạp chí Nghiên cứu Y học · doi:10.52852/tcncyh.v204i7.5265
Sievinen, S. (2026) · Aaltodoc (Aalto University)
Tong, W., Liu, X., & Zhou, J. (2026) · International Conference on Software Engineering and Computer Applications · doi:10.1145/3829641.3829680
“168 Zhiyuan Wan, 2021 [5] Differences between machine learning development and traditional software development.”
Kiviniemi, A. (2024) · UEF eRepo (University of Eastern Finland)
Automated machine learning to foster AI development processes: an explorative study
Polzer, A. K., & Thalmann, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1867883
Metadata
- Authors
- Wan, Z., Xia, X., Lo, D., & Murphy, G. C.
- Affiliations
- Zhejiang University (CN)
- University of British Columbia (CA)
- Monash University (AU)
- Singapore Management University (SG)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2020
- Pages
- 1-1
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2970433196
- Cited by
- 181
- References
- 31
- Citations by year
- 2026: 11 · 2025: 28 · 2024: 31 · 2023: 35 · 2022: 36 · 2021: 29 · 2020: 10
- FWCI
- 23.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion)
- Funders
- National Basic Research Program of China (2018YFB1003904)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- machine learning systems, software development practices, software engineering process, requirements engineering, software design, software testing, skill variety, problem solving, task identity, job characteristics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.