An integration engineering framework for machine learning in healthcare
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Background and Objectives: Machine Learning offers opportunities to improve patient outcomes, team performance, and reduce healthcare costs. Yet only a small fraction of all Machine Learning models for health care have been successfully integrated into the clinical space. There are no current guidelines for clinical model integration, leading to waste, unnecessary costs, patient harm, and decreases in efficiency when improperly implemented. Systems engineering is widely used in industry to achieve an integrated system of systems through an interprofessional collaborative approach to system design, development, and integration. We propose a framework based on systems engineering to guide the development and integration of Machine Learning models in healthcare. Methods: Applied systems engineering, software engineering and health care Machine Learning software development practices were reviewed and critically appraised to establish an understanding of limitations and challenges within these domains. Principles of systems engineering were used to develop solutions to address the identified problems. The framework was then harmonized with the Machine Learning software development process to create a systems engineering-based Machine Learning software development approach in the healthcare domain. Results: We present an integration framework for healthcare Artificial Intelligence that considers the entirety of this system of systems. Our proposed framework utilizes a combined software and integration engineering approach and consists of four phases: (1) Inception, (2) Preparation, (3) Development, and (4) Integration. During each phase, we present specific elements for consideration in each of the three domains of integration: The Human, The Technical System, and The Environment. There are also elements that are considered in the interactions between these domains. Conclusion: Clinical models are technical systems that need to be integrated into the existing system of systems in health care. A systems engineering approach to integration ensures appropriate elements are considered at each stage of model design to facilitate model integration. Our proposed framework is based on principles of systems engineering and can serve as a guide for model development, increasing the likelihood of successful Machine Learning translation and integration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579 · cited 4×
“) often do not map directly to clinical performance indicators (14, 37) [C11] - Need model transparency (11, 39, 41) [C12]”
BackgroundInfluentialKey challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246
Do no harm: a roadmap for responsible machine learning for health care
Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0548-6 · Cited by 1,266
Self-citationLee, D., & Yoon, S. N. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18010271 · Cited by 1,017
Next-generation phenotyping of electronic health records
Hripcsak, G., & Albers, D. J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001145 · Cited by 795
Aung, Y. Y. M., Wong, D. C. S., & Ting, D. S. W. (2021) · British Medical Bulletin · doi:10.1093/bmb/ldab016 · Cited by 511 · cited 3×
“) often do not map directly to clinical performance indicators (14, 37) [C11] - Need model transparency (11, 39, 41) [C12]”
BackgroundMethodYoung, A. T., Amara, D., Bhattacharya, A., & Wei, M. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00132-1 · Cited by 400
With an eye to AI and autonomous diagnosis
Keane, P. A., & Topol, E. J. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0048-y · Cited by 304 · cited 2×
“Frontiers in Digital Health 10 system are organized to achieve the purpose of the system (9, 10).”
BackgroundHow does Machine Learning Change Software Development Practices?
Wan, Z., Xia, X., Lo, D., & Murphy, G. C. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2937083 · Cited by 181 · cited 3×
“Frontiers in Digital Health 04 development and deployment of Machine Learning models, they have been difficult and expensive to maintain and integrate (35).”
BackgroundInfluential
Cited by
Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad088 · Cited by 109
Machine Learning Operations in Health Care: A Scoping Review
Rajagopal, A., Ayanian, S., Ryu, A. J., Qian, R., Legler, S. R., Peeler, E. A., Issa, M., Coons, T. J., & Kawamoto, K. (2024) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2024.06.009 · Cited by 52
Addressing ethical issues in healthcare artificial intelligence using a lifecycle-informed process
Collins, B. X., Bélisle-Pipon, J.-C., Evans, B. J., Ferryman, K., Jiang, X., Nebeker, C., Novak, L., Roberts, K., Were, M., Yin, Z., et al. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae108 · Cited by 21
A scoping review of silent trials for medical artificial intelligence
Tikhomirov, L., Semmler, C., Prizant, N., Bhasin, S., Kenyon, G., van der Vegt, A., Erdman, L., Kurian, N. C., Thompson, H., Palmer, L. J., et al. (2026) · Nature Health · doi:10.1038/s44360-025-00048-z · Cited by 17
Self-citationOwoyemi, A., Osuchukwu, J., Salwei, M. E., & Boyd, A. (2025) · JMIRx Med · doi:10.2196/65565 · Cited by 18
Metadata
- Authors
- Assadi, A., Laussen, P. C., Goodwin, A. J., Goodfellow, S., Dixon, W., Greer, R. W., Jegatheeswaran, A., Singh, D., McCradden, M., Gallant, S. N., Goldenberg, A., Eytan, D., & Mazwi, M. L.
- Venue
- Frontiers in Digital Health
- Publisher
- Frontiers Media SA
- Published
- 2022-08-04
- Volume
- 4
- Pages
- 932411-932411
- ISSN
- 2673-253X (ISSN-L)
- OpenAlex
- W4289730999
- Cited by
- 20
- References
- 11
- FWCI
- 0.82
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An integration engineering framework for machine learning in healthcare. https://app.cochat.ai/papers/10.3389/fdgth.2022.932411
- The paper
- Assadi, A., Laussen, P. C., Goodwin, A. J., Goodfellow, S., Dixon, W., Greer, R. W., Jegatheeswaran, A., Singh, D., McCradden, M., Gallant, S. N., Goldenberg, A., Eytan, D., & Mazwi, M. L. (2022). An integration engineering framework for machine learning in healthcare. Frontiers in Digital Health. https://doi.org/10.3389/fdgth.2022.932411
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.