The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Software Engineering for Machine Learning: A Case Study
Abstract
Recent advances in machine learning have stimulated widespread interest within the Information Technology sector on integrating AI capabilities into software and services. This goal has forced organizations to evolve their development processes. We report on a study that we conducted on observing software teams at Microsoft as they develop AI-based applications. We consider a nine-stage workflow process informed by prior experiences developing AI applications (e.g., search and NLP) and data science tools (e.g. application diagnostics and bug reporting). We found that various Microsoft teams have united this workflow into preexisting, well-evolved, Agile-like software engineering processes, providing insights about several essential engineering challenges that organizations may face in creating large-scale AI solutions for the marketplace. We collected some best practices from Microsoft teams to address these challenges. In addition, we have identified three aspects of the AI domain that make it fundamentally different from prior software application domains: 1) discovering, managing, and versioning the data needed for machine learning applications is much more complex and difficult than other types of software engineering, 2) model customization and model reuse require very different skills than are typically found in software teams, and 3) AI components are more difficult to handle as distinct modules than traditional software components - models may be "entangled" in complex ways and experience non-monotonic error behavior. We believe that the lessons learned by Microsoft teams will be valuable to other organizations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
The KDD process for extracting useful knowledge from volumes of data
Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996) · Communications of the ACM · doi:10.1145/240455.240464 · Cited by 1,934
DevOps Capabilities, Practices, and Challenges: Insights from a Case Study
Senapathi, M., Buchan, J., & Osman, H. (2018) · Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018 · doi:10.1145/3210459.3210465 · Cited by 172
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Nushi, B., Kamar, E., & Horvitz, E. (2018) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v6i1.13337 · Cited by 126
Self-citation
Cited by
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Beyond accuracy: behavioral testing of NLP models with CheckList
Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.442 · Cited by 890
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Challenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638
Data collection and quality challenges in deep learning: a data-centric AI perspective
Whang, S. E., Roh, Y., Song, H., & Lee, J.-G. (2023) · The VLDB Journal · doi:10.1007/s00778-022-00775-9 · Cited by 573
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“Moreover, various techniques researched for this stage can contribute to the trustworthiness of an AI system [17].”
BackgroundFeng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423
Perifanis, N.-A., & Kitsios, F. (2023) · Information · doi:10.3390/info14020085 · Cited by 407
Hutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
Lwakatare, L. E., Raj, A., Crnkovic, I., Bosch, J., & Olsson, H. H. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106368 · Cited by 241
Toward AI Governance: Identifying Best Practices and Potential Barriers and Outcomes
Papagiannidis, E., Enholm, I. M., Dremel, C., Mikalef, P., & Krogstie, J. (2023) · Information Systems Frontiers · doi:10.1007/s10796-022-10251-y · Cited by 191
How does Machine Learning Change Software Development Practices?
Wan, Z., Xia, X., Lo, D., & Murphy, G. C. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2937083 · Cited by 181
Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157
“These stages are as follows [2, 24, 25]: We present our operationalised ethical AI framework across these three stages, providing actionable solutions to prevent, mitigate and address ethical issues.”
BackgroundMethodFloridi, L., Holweg, M., Taddeo, M., Amaya, J., Mökander, J., & Wen, Y. (2022) · doi:10.2139/ssrn.4064091 · Cited by 157
How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Piorkowski, D., Park, S., Wang, A. Y., Wang, D., Muller, M., & Portnoy, F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449205 · Cited by 151
Visual tools for teaching machine learning in K-12: A ten-year systematic mapping
Gresse von Wangenheim, C., Hauck, J. C. R., Pacheco, F. S., & Bertonceli Bueno, M. F. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10570-8 · Cited by 130
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126
AI-driven innovations in software engineering: a review of current practices and future directions
Alenezi, M., & Akour, M. (2025) · Applied Sciences · doi:10.3390/app15031344 · Cited by 121
Understanding and Visualizing Data Iteration in Machine Learning
Hohman, F., Wongsuphasawat, K., Kery, M. B., & Patel, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376177 · Cited by 110
Engineering AI Systems: A Research Agenda
Bosch, J., Olsson, H. H., & Crnkovic, I. (2021) · Advances in Systems Analysis, Software Engineering, and High Performance Computing · doi:10.4018/978-1-7998-5101-1.ch001 · Cited by 108
Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Xin, D., Wu, E. Y., Lee, D. J.-L., Salehi, N., & Parameswaran, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445306 · Cited by 110
The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality
Gordon, M. L., Zhou, K., Patel, K., Hashimoto, T., & Bernstein, M. S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445423 · Cited by 101
Balayn, A., Lofi, C., & Houben, G.-J. (2021) · The VLDB Journal · doi:10.1007/s00778-021-00671-8 · Cited by 98
Big Data Systems: A Software Engineering Perspective
Davoudian, A., & Liu, M. (2021) · ACM Computing Surveys · doi:10.1145/3408314 · Cited by 88
Johnson, M., Albizri, A., & Harfouche, A. (2023) · Information Systems Frontiers · doi:10.1007/s10796-021-10137-5 · Cited by 88
An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry
Jiang, W., Synovic, N., Hyatt, M., Schorlemmer, T. R., Sethi, R., Lu, Y.-H., Thiruvathukal, G. K., & Davis, J. C. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00206 · Cited by 76
Generative Artificial Intelligence for Software Engineering - a Research Agenda
NguyenDuc, A., Cabrero-Daniel, B., Arora, C., Przybylek, A., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K.-K., et al. (2023) · doi:10.2139/ssrn.4622517 · Cited by 71
Adoption of Machine Learning Systems for Medical Diagnostics in Clinics: Qualitative Interview Study
Pumplun, L., Fecho, M., Wahl, N., Peters, F., & Buxmann, P. (2021) · Journal of Medical Internet Research · doi:10.2196/29301 · Cited by 57
Generative Artificial Intelligence for Software Engineering—A Research Agenda
Nguyen‐Duc, A., Cabrero‐Daniel, B., Przybylek, A., Arora, C., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K., et al. (2025) · Software: Practice and Experience · doi:10.1002/spe.70005 · Cited by 60
Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
Wang, Z. J., Kulkarni, C., Wilcox, L., Terry, M., & Madaio, M. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642335 · Cited by 55
Taxonomy of Machine Learning Safety: A Survey and Primer
Mohseni, S., Wang, H., Xiao, C., Yu, Z., Wang, Z., & Yadawa, J. (2023) · ACM Computing Surveys · doi:10.1145/3551385 · Cited by 53
Human-Centered AI Design in Reality: A Study of Developer Companies’ Practices
Hartikainen, M., Väänänen, K., Lehtiö, A., Ala-Luopa, S., & Olsson, T. (2022) · Nordic Human-Computer Interaction Conference · doi:10.1145/3546155.3546677 · Cited by 48
Menzies, T. (2020) · IEEE Software · doi:10.1109/ms.2019.2954841 · Cited by 43
AI Regulation Is (not) All You Need
Lucaj, L., van der Smagt, P., & Benbouzid, D. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594079 · Cited by 40
Interactive Debugging and Steering of Multi-Agent AI Systems
Epperson, W., Bansal, G., Dibia, V. C., Fourney, A., Gerrits, J., Zhu, E. (., & Amershi, S. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713581 · Cited by 35
Self-citationCapturing end-to-end provenance for machine learning pipelines
Schlegel, M., & Sattler, K.-U. (2025) · Information Systems · doi:10.1016/j.is.2024.102495 · Cited by 32
Data excellence for AI: why should you care?
Aroyo, L., Lease, M., Paritosh, P., & Schaekermann, M. (2022) · Interactions · doi:10.1145/3517337 · Cited by 31
From a Data Science Driven Process to a Continuous Delivery Process for Machine Learning Systems
Lwakatare, L. E., Crnkovic, I., Rånge, E., & Bosch, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-64148-1_12 · Cited by 28
Dragomirescu, O.-A., Crăciun, P.-C., & Bologa, A. R. (2025) · Systems · doi:10.3390/systems13020087 · Cited by 22
Voronov, G., Lightheart, R., Frandsen, A., Bargh, B., Haynes, S. E., Spencer, E., Schoenhardt, K. E., Davidson, C., Schaum, A., Macherla, V. R., et al. (2022) · doi:10.1101/2022.10.09.511482 · Cited by 18
MLflow2PROV: Extracting Provenance from Machine Learning Experiments
Schlegel, M., & Sattler, K.-U. (2023) · Proceedings of the Seventh Workshop on Data Management for End-to-End Machine Learning · doi:10.1145/3595360.3595859 · Cited by 17
Tan, M., Lee, H., Wang, D., & Subramonyam, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637358 · Cited by 14
Component Mismatches Are a Critical Bottleneck to Fielding AI-Enabled Systems in the Public Sector
Lewis, G. A., Bellomo, S., & Galyardt, A. (2019) · arXiv · doi:10.48550/arxiv.1910.06136 · Cited by 8
Towards Role definition in agile ai-based system development: perspectives and reflections
Souza, F. C. M., Souza, A. C. C., Amorim, B. F., & Cordeiro, T. D. (2024) · Proceedings of the XXIII Brazilian Symposium on Software Quality · doi:10.1145/3701625.3701661 · Cited by 3
MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery
Du, S., Yan, X., Shi, J., Cao, Z., Feng, S., Liang, Z., Sun, B., Guo, X., Peng, T., Zhou, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.06473 · Cited by 31
Who Owns AI When It Breaks? The Convergence of AI Governance and Cybersecurity Accountability
Montgomery, J., & Copeland, J. (2026) · The Pinnacle: A Journal by Scholar-Practitioners · doi:10.61643/c79060
Metadata
- Authors
- Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T.
- Venue
- 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 291-300
- OpenAlex
- W2922234936
- Cited by
- 1,135
- References
- 5
- FWCI
- 92.62
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
