ml-experiment: A Python framework for reproducible data science
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Nowadays, data science projects are usually developed in an unstructured way, which makes it difficult to reproduce. It is also hard to move from an experimental environment to production. Operational workflows such as containerization, continuous deployment, and cloud orchestration allow data science researchers to move a pipeline from a local environment to the cloud. Being aware of the difficulties of setting those workflows up, this paper presents a framework to ease experiment tracking and operationalizing machine learning by combining existent and well-supported technologies. These technologies include Docker, Mlflow, Ray, among others. The framework provides an opinionated workflow to design and execute experiments either on a local environment or the cloud. ml-experiment includes: an automatic tracking system for the most famous machine learning libraries: Tensorflow, Keras, Fastai, Xgboost and Lightgdm, first-class support for distributed training and hyperparameter optimization, and a Command Line Interface (CLI) for packaging and running projects inside containers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2011) · Cited by 3,939
An introduction to Docker for reproducible research
Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059 · cited 1×
“Packaging Engine is the last module, which is responsible for generating a shareable version of the project using Docker [9] technology to create an image of all dependencies.”
MethodArtificial intelligence faces reproducibility crisis
Hutson, M. (2018) · Science · doi:10.1126/science.359.6377.725 · Cited by 564
An introduction to Docker for reproducible research, with examples from the R environmentPreprint
Boettiger, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0846 · Cited by 372 · cited 1×
“Packaging Engine is the last module, which is responsible for generating a shareable version of the project using Docker [9] technology to create an image of all dependencies.”
MethodTen simple rules for responsible big data research
Zook, M., Barocas, S., boyd, D., Crawford, K., Keller, E., Gangadharan, S. P., Goodman, A., Hollander, R., Koenig, B. A., Metcalf, J., et al. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005399 · Cited by 343
Provenance Analysis for Missing Answers and Integrity Repairs.
Zaharia, M., Chen, A., Davidson, A., Ghodsi, A., Hong, S. A., Konwinski, A., Murching, S., Nykodym, T., Ogilvie, P., Parkhe, M., et al. (2018) · IEEE Data(base) Engineering Bulletin · Cited by 112
Reproducibility of Data-Oriented Experiments in e-Science (Dagstuhl Seminar 16041)
Freire, J., Fuhr, N., & Rauber, A. (2016) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dagrep.6.1.108 · Cited by 67 · cited 1×
“Introduction Reproducibility is a challenge in modern research and produces a lot of discussion [1, 2, 3].”
BackgroundProceedings of the 2012 ACM SIGMOD International Conference on Management of Data
Candan, K. S., Chen, Y., Snodgrass, R. T., Gravano, L., & Fuxman, A. D. (2012) · Cited by 10
Good Enough Practices in Scientific Computing
Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., & Teal, T. K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005510 · Cited by 8 · cited 1×
“Among all types of reproducible research, the work presented in this paper focuses on computational research [4]: building a set of tools and a specific workflow based on the principles of version control, automation, tracking, and environment isolation [5, 6].”
Background
Research works printed but not linked: 1
- Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, Fen Xie, and Corey Zumar. Accelerating the machine learning lifecycle with mlflow. IEEE Data Eng. Bull., 41:39-45, 2018.
Cited by
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Kreuzberger, D., Kühl, N., & Hirschl, S. (2023) · IEEE Access · doi:10.1109/access.2023.3262138 · Cited by 766
“Reproducibility is the ability to reproduce an ML experiment and obtain the exact same results [14,32,40,46] [α, β, δ, ε, η].”
MethodInfluentialEmployability prediction: a survey of current approaches, research challenges and applications
Mezhoudi, N., Alghamdi, R., Aljunaid, R., Krishnasamy, G., & Düştegör, D. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03276-9 · Cited by 106
“Reproducibility is a challenge in applied data science and produces a lot of discussion (Molner Domenech and Guillén 2020).”
BackgroundA Comprehensive Review on Scaling Machine Learning Workflows Using Cloud Technologies and DevOps
Ramesh, G., Pai, T. V., Birau, R., Poojary, K. K., Abhay, Shingad, A. R., Sowjanya, N., Popescu, V., Mitroi, A. T., Chireac, R. M. N., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3599281 · Cited by 40
“Relevance: This evaluates the machine learning model’s prediction speed, which is crucial for determining how responsive a system is in real-world settings [58].”
A Model-Driven Approach for Systematic Reproducibility and Replicability of Data Science Projects
Melchor, F., Rodriguez-Echeverria, R., Conejero, J. M., Prieto, Á. E., & Gutiérrez, J. D. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-07472-1_9 · Cited by 6
DevOps Practices Into Machine Learning
Jain, S., & Kumar, P. (2024) · IEEE International Conference on Intelligent Systems, Smart and Green Technologies (ICISSGT) · doi:10.1109/icissgt58904.2024.00029 · Cited by 7
Metadata
- Authors
- Molner Domenech, A., & Guillén, A.
- Affiliations
- Universidad de Granada (ES)
- Corresponding author
- Guillén, Alberto
- ORCID
- Guillén, Alberto
- Venue
- Journal of Physics: Conference Series
- Publisher
- IOP Publishing
- Published
- 2020-09-01
- Volume
- 1603
- Issue
- 1
- Pages
- 012025
- ISSN
- 1742-6588 (ISSN-L)
- 1742-6596
- OpenAlex
- W3087897998
- Cited by
- 10
- References
- 9
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 2 · 2022: 2 · 2021: 2
- FWCI
- 3.21
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/3.0/ (vor)
- iopscience.iop.org/info/page/text-and-data-mining (tdm)
- Locations
- Journal of Physics Conference Series (publishedVersion, cc-by)
- Funders
- Ministerio de Economía y Competitividad (FPA2017-85197-P)
- European Regional Development Fund
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- reproducible machine learning, experiment tracking, MLflow, Docker containerization, rays, machine learning pipelines, hyperparameter optimization, distributed training, cloud deployment, continuous deployment, command-line interface, Python framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ml-experiment: A Python framework for reproducible data science. https://app.cochat.ai/papers/10.1088/1742-6596/1603/1/012025
- The paper
- Molner Domenech, A., & Guillén, A. (2020). ml-experiment: A Python framework for reproducible data science. Journal of Physics: Conference Series. https://doi.org/10.1088/1742-6596/1603/1/012025
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.