CoChat
Sign Up
Cited by 130Software Engineering Techniques and PracticesSoftware System Performance and ReliabilitySoftware Engineering Research

Towards MLOps: A Framework and Maturity Model

John, Meenu Mary, Olsson, Helena Holmstrom, Bosch, Jan·2021·2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)
DOI 10.1109/seaa53835.2021.00050
Open this paper in CoChat — free account

Abstract

The adoption of continuous software engineering practices such as DevOps (Development and Operations) in business operations has contributed to significantly shorter software development and deployment cycles. Recently, the term MLOps (Machine Learning Operations) has gained increasing interest as a practice that brings together data scientists and operations teams. However, the adoption of MLOps in practice is still in its infancy and there are few common guidelines on how to effectively integrate it into existing software development practices. In this paper, we conduct a systematic literature review and a grey literature review to derive a framework that identifies the activities involved in the adoption of MLOps and the stages in which companies evolve as they become more mature and advanced. We validate this framework in three case companies and show how they have managed to adopt and integrate MLOps in their large-scale software development companies. The contribution of this paper is threefold. First, we review contemporary literature to provide an overview of the state-of-the-art in MLOps. Based on this review, we derive an MLOps framework that details the activities involved in the continuous development of machine learning models. Second, we present a maturity model in which we outline the different stages that companies go through in evolving their MLOps practices. Third, we validate our framework in three embedded systems case companies and map the companies to the stages in the maturity model.

Integrity signals

Registry

Crossref · 10.1109/seaa53835.2021.00050

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • John et al. [11] derived a framework for MLOps that shows the activities that take part in the continuous development of ML models and maps the practices of companies to MLOps maturity model to show how practices evolve through stages.

    “More specifically, the authors derived a framework for MLOps that shows the activities that take part in the continuous development of ML models and maps the practices of companies to MLOps maturity model to show how practices evolve through stages.”

    From A Multivocal Review of MLOps Practices, Challenges and Open Issues (2026) · cites this paper as [11]

  • John et al. [11] delivered an MLR on the adoption of MLOps in practice.

    “John et al. [11] delivered an MLR on the adoption of MLOps in practice.”

    From A Multivocal Review of MLOps Practices, Challenges and Open Issues (2026) · cites this paper as [11]

  • The authors of [24] did not study tools and infrastructures for operationalizing ML models.

    “However, the authors of [24] did not study tools and infrastructures for operationalizing ML models.”

    From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [24]

  • [24] concludes that successful AI/ML operationalization ensures proper model deployment and updating in different environments.

    “[24] concludes that successful AI/ML operationalization ensures proper model deployment and updating in different environments.”

    From Operationalizing machine learning models: a systematic literature review (2022) · cites this paper as [24]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
John, M. M., Olsson, H. H., & Bosch, J.
Affiliations
  • Malmö University (SE)
  • Chalmers University of Technology (SE)
ORCID
Venue
2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)
Publisher
IEEE
Published
2021-09
Pages
1-8
DOI
10.1109/seaa53835.2021.00050
OpenAlex
W3210418146
Cited by
130
References
31
Citations by year
2026: 12 · 2025: 29 · 2024: 35 · 2023: 31 · 2022: 22 · 2021: 1
FWCI
18.42
Type
conference paper
Language
en
Open access
closed
Licence
Locations
  • Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (publishedVersion)
  • Chalmers Research (Chalmers University of Technology) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
MLOps, maturity model, DevOps, software development practices, embedded systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review