CoChat
Sign Up
Cited by 31Software Engineering Techniques and PracticesScientific Computing and Data ManagementSoftware System Performance and Reliability

Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?

Open access
Makinen, Sasu, Skogstrom, Henrik, Laaksonen, Eero, Mikkonen, Tommi·2021·2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)
DOI 10.1109/wain52551.2021.00024
Open this paper in CoChat — free accountOpen PDF

Abstract

Following continuous software engineering practices, there has been an increasing interest in rapid deployment of machine learning (ML) features, called MLOps. In this paper, we study the importance of MLOps in the context of data scientists' daily activities, based on a survey where we collected responses from 331 professionals from 63 different countries in ML domain, indicating on what they were working on in the last three months. Based on the results, up to 40% respondents say that they work with both models and infrastructure; the majority of the work revolves around relational and time series data; and the largest categories of problems to be solved are predictive analysis, time series data, and computer vision. The biggest perceived problems revolve around data, although there is some awareness of problems related to deploying models to production and related procedures. To hypothesise, we believe that organisations represented in the survey can be divided to three categories - (i) figuring out how to best use data; (ii) focusing on building the first models and getting them to production; and (iii) managing several models, their versions and training datasets, as well as retraining and frequent deployment of retrained models. In the results, the majority of respondents are in category (i) or (ii), focusing on data and models; however the benefits of MLOps only emerge in category (iii) when there is a need for frequent retraining and redeployment. Hence, setting up an MLOps pipeline is a natural step to take, when an organization takes the step from ML as a proof-of-concept to ML as a part of nominal activities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/wain52551.2021.00024

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The large number of artifacts requires strong governance.

    “that require a strong governance [27], [32], [45]”

    From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [32]

    The Conclusions (page unavailable) say that as the number of models grows, maintaining coherence and quality becomes more complex and requires systematic versioning of models and data sets. This supports a need to manage growing ML assets, but the claim broadens that specific requirement to strong governance of artifacts generally. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • There is a lack of highly skilled experts for MLOps roles, especially architects, data engineers, ML engineers, and DevOps engineers.

    “As our identified sources point out, there is a lack of highly skilled experts for these rolesespecially with regard to architects, data engineers, ML engineers, and DevOps engineers [26], [32], [38], [α, ε].”

    From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [32]

    The Survey section lists “not enough engineers/DevOps personnel” as an obstacle respondents were asked to rate, and the Figure 6 caption identifies it as a perceived challenge. The cited paper does not establish a shortage of highly skilled experts or single out architects, data engineers, and ML engineers as roles with shortages. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Versioning of data, model, and code is required to ensure robustness and reproducibility.

    “as well as versioning of data, model, and code to ensure robustness and reproducibility [7], [32], [39].”

    From Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2023) · cites this paper as [32]

    The Conclusions say that maintaining coherence and quality as models multiply requires systematic versioning of models and datasets. Background and Motivation describes reproducible increments involving code, data, and models. Our copy does not explicitly require versioning code or say that versioning ensures robustness and reproducibility. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [4]4. P Debois. Devops: A software revolution in the making. Cutter IT Journal, 24(8), 2011.

Cited by

Sorted by citations

Related

Authors
Makinen, S., Skogstrom, H., Laaksonen, E., & Mikkonen, T.
Affiliations
University of Helsinki (FI)
ORCID
Mikkonen, Tommi
Venue
2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)
Publisher
IEEE
Published
2021-05
Pages
109-112
DOI
10.1109/wain52551.2021.00024
OpenAlex
W3136912235
Cited by
28
References
10
Citations by year
2025: 6 · 2024: 6 · 2023: 5 · 2022: 7 · 2021: 4
FWCI
4.56
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) (publishedVersion)
  • Työväentutkimus Vuosikirja (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Business Finland
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
Keywords
MLOps, data scientists, machine learning deployment, model versioning, model retraining, continuous software engineering, time series, predictive analytics, computer vision

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review