The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Capturing end-to-end provenance for machine learning pipelines
Abstract
Modern workflows for developing ML pipelines utilize ML artifact management systems (ML AMSs) such as MLflow in addition to traditional version control systems such as Git. ML AMSs collect data, model, metadata and software artifacts used and produced in pipeline development workflows. While ensuring repeatability and reproducibility, the provenance capabilities are still rudimentary, mainly due to incomplete traces, coarse granularity, and limited query capabilities. In this paper, we introduce a comprehensive PROV-compliant provenance model that captures end-to-end provenance traces of ML pipelines, their artifacts, and their relationships based on MLflow and Git activities. Moreover, we present the tool MLflow2PROV for continuously extracting provenance graphs according to our model, enabling querying, analyzing, and processing of the collected provenance information.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Agile Software Development, Principles, Patterns, and Practices
Martin, R. (2002) · Cited by 1,466
Modeling wine preferences by data mining from physicochemical properties
Cortez, P., Cerdeira, A., Almeida, F., Matos, T., & Reis, J. (2009) · Decision Support Systems · doi:10.1016/j.dss.2009.05.016 · Cited by 1,273
OpenML: networked science in machine learning
Vanschoren, J., van Rijn, J. N., Bischl, B., & Torgo, L. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2641190.2641198 · Cited by 1,141
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135
Packaging research artefacts with RO-Crate
Soiland‐Reyes, S., Sefton, P., Crosas, M., Castro, L. J., Coppens, F., Fernández, J. M., Garijo, D., Grüning, B. A., Rosa, M. L., Leo, S., et al. (2022) · Data Science · doi:10.3233/ds-210053 · Cited by 280
SKOS Simple Knowledge Organization System Primer
Isaac, A., & Summers, E. (2009) · Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) · Cited by 172
PROV-Overview. An Overview of the PROV Family of Documents
Groth, P. T., & Moreau, L. (2013) · Cited by 113
Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle
Chen, A., Chow, A., Davidson, A., DCunha, A., Ghodsi, A., Hong, S. A., Konwinski, A., Mewald, C., Murching, S., Nykodym, T., et al. (2020) · Proceedings of the Fourth International Workshop on Data Management for End-to-End Machine Learning · doi:10.1145/3399579.3399867 · Cited by 112
Management of Machine Learning Lifecycle Artifacts: A Survey
Schlegel, M., & Sattler, K.-U. (2023) · ACM SIGMOD Record · doi:10.1145/3582302.3582306 · Cited by 93
Self-citationRecording provenance of workflow runs with RO-Crate
Leo, S., Crusoe, M. R., Rodríguez‐Navas, L., Sirvent, R., Kanitz, A., Geest, P. F. G. D., Wittner, R., Pireddu, L., Garijo, D., Fernández, J. M., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309210 · Cited by 76
Vamsa: Automated Provenance Tracking in Data Science Scripts
Namaki, M. H., Floratou, A., Psallidas, F., Krishnan, S., Agrawal, A., Wu, Y., Zhu, Y., & Weimer, M. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403205 · Cited by 59
Esteves, D., Moussallem, D., Neto, C. B., Soru, T., Usbeck, R., Ackermann, M. R., & Lehmann, J. (2015) · International Conference on Semantic Systems · doi:10.1145/2814864.2814883 · Cited by 55
Workflow provenance in the lifecycle of scientific machine learning
Souza, R., Azevedo, L. G., Lourenço, V., Soares, E. F. D. S., Thiago, R. M., Brandão, R., Civitarese, D., Brazil, E. V., Moreno, M. F., Valduriez, P., et al. (2021) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.6544 · Cited by 46
Samuel, S., & König‐Ries, B. (2022) · Journal of Biomedical Semantics · doi:10.1186/s13326-021-00253-1 · Cited by 26
MLINSPECT: A Data Distribution Debugger for Machine Learning Pipelines
Grafberger, S., Guha, S., Stoyanovich, J., & Schelter, S. (2021) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/3448016.3452759 · Cited by 24
Machine learning in the real world
Chaoji, V., Rastogi, R. R., & Roy, G. (2016) · Proceedings of the VLDB Endowment · doi:10.14778/3007263.3007318 · Cited by 19
MLflow2PROV: Extracting Provenance from Machine Learning Experiments
Schlegel, M., & Sattler, K.-U. (2023) · Proceedings of the Seventh Workshop on Data Management for End-to-End Machine Learning · doi:10.1145/3595360.3595859 · Cited by 17
Self-citationAnalyzing Software Engineering Processes with Provenance-based Knowledge Graphs
Schreiber, A., von Kurnatowski, L., & de Boer, C. (2021) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero50100.2021.9438358 · Cited by 7
Demonstration of Geyser: Provenance Extraction and Applications over Data Science Scripts
Psallidas, F., Leszczynski, M. E., Namaki, M. H., Floratou, A., Agrawal, A., Karanasos, K., Krishnan, S., Subotic, P., Weimer, M., Wu, Y., et al. (2023) · Companion of the 2023 International Conference on Management of Data · doi:10.1145/3555041.3589717 · Cited by 3
Recording provenance of workflow runs with RO-Crate (RO-Crate and mapping)
Leo, S., Crusoe, M. R., Rodríguez‐Navas, L., Sirvent, R., Kanitz, A., Geest, P. F. G. D., Wittner, R., Pireddu, L., Garijo, D., Fernández, J. M., et al. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.10368989 · Cited by 2
Research works printed but not linked: 5
- Zaharia, 2018, Accelerating the Machine Learning Lifecycle with MLflow, IEEE Data Eng. Bull., 41, 39
- Huynh, 2014, ProvStore: A Public Provenance Repository, vol. 8628, 275
- Schlegel, 2023, Extracting Provenance of Machine Learning Experiment Pipeline Artifacts, vol. 13985, 238
- Chapman, 2022, DPDS: Assisting Data Science with Data Provenance, PVLDB, 15, 3614
- S. Schelter, J.-H. Böse, J. Kirschnick, T. Klein, S. Seufert, Automatically Tracking Metadata and Provenance of Machine Learning Experiments, in: MLSys@NIPS ’17, 2017, pp. 1–8.
Metadata
- Authors
- Schlegel, M., & Sattler, K.-U.
- Venue
- Information Systems
- Publisher
- Elsevier BV
- Published
- 2025-07
- Volume
- 132
- Pages
- 102495
- ISSN
- 0306-4379
- Cited by
- 32
- References
- 21
- Language
- en
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Funders
- Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft (5575/10-3)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
