Ten Simple Rules for Reproducible Computational Research
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Replication is the cornerstone of a cumulative science [1]. However, new tools and technologies, massive amounts of data, interdisciplinary approaches, and the complexity of the questions being asked are complicating replication efforts, as are increased pressures on scientists to advance their research [2]. As full replication of studies on independently collected data is often not feasible, there has recently been a call for reproducible research as an attainable minimum standard for assessing the value of scientific claims [3]. This requires that papers in experimental science describe the results and provide a sufficiently clear protocol to allow successful repetition and extension of analyses based on original data [4]. The importance of replication and reproducibility has recently been exemplified through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], and through a high number of failing clinical trials [8], [9]. This has led to discussions on how individual researchers, institutions, funding bodies, and journals can establish routines that increase transparency and reproducibility. In order to foster such aspects, it has been suggested that the scientific community needs to develop a “culture of reproducibility” for computational science, and to require it for published claims [3]. We want to emphasize that reproducibility is not only a moral responsibility with respect to the scientific field, but that a lack of reproducibility can also be a burden for you as an individual researcher. As an example, a good practice of reproducibility is necessary in order to allow previously developed methodology to be effectively applied on new data, or to allow reuse of code and results for new projects. In other words, good habits of reproducibility may actually turn out to be a time-saver in the longer run. We further note that reproducibility is just as much about the habits that ensure reproducible research as the technologies that can make these processes efficient and realistic. Each of the following ten rules captures a specific aspect of reproducibility, and discusses what is needed in terms of information handling and tracking of procedures. If you are taking a bare-bones approach to bioinformatics analysis, i.e., running various custom scripts from the command line, you will probably need to handle each rule explicitly. If you are instead performing your analyses through an integrated framework (such as GenePattern [10], Galaxy [11], LONI pipeline [12], or Taverna [13]), the system may already provide full or partial support for most of the rules. What is needed on your part is then merely the knowledge of how to exploit these existing possibilities. In a pragmatic setting, with publication pressure and deadlines, one may face the need to make a trade-off between the ideals of reproducibility and the need to get the research out while it is still relevant. This trade-off becomes more important when considering that a large part of the analyses being tried out never end up yielding any results. However, frequently one will, with the wisdom of hindsight, contemplate the missed opportunity to ensure reproducibility, as it may already be too late to take the necessary notes from memory (or at least much more difficult than to do it while underway). We believe that the rewards of reproducibility will compensate for the risk of having spent valuable time developing an annotated catalog of analyses that turned out as blind alleys. As a minimal requirement, you should at least be able to reproduce the results yourself. This would satisfy the most basic requirements of sound research, allowing any substantial future questioning of the research to be met with a precise explanation. Although it may sound like a very weak requirement, even this level of reproducibility will often require a certain level of care in order to be met. There will for a given analysis be an exponential number of possible combinations of software versions, parameter values, pre-processing steps, and so on, meaning that a failure to take notes may make exact reproduction essentially impossible. With this basic level of reproducibility in place, there is much more that can be wished for. An obvious extension is to go from a level where you can reproduce results in case of a critical situation to a level where you can practically and routinely reuse your previous work and increase your productivity. A second extension is to ensure that peers have a practical possibility of reproducing your results, which can lead to increased trust in, interest for, and citations of your work [6], [14]. We here present ten simple rules for reproducibility of computational research. These rules can be at your disposal for whenever you want to make your researc
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rough consensus reduces to three ingredients.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While there have been a number of recent, often domain-focused publications advocating for specific improvements in practices relating to data management and archival1,11,12, FAIR differs in that it describes concise, domain-independent, high-level principles that can be applied to a wide range of scholarly outputs.”
From The FAIR Guiding Principles for scientific data management and stewardship (2016) · cites this paper as [12]
In the citing paper
“In a recent publication reviewing the automated citation screening methods reported for systematic reviews, the number of studies that met the current standards for reproducible software engineering experiments was low [29, 30, 36]; of 33 studies reporting approaches to automated screening, no study provided all the criteria for reproducible reporting [36].”
From A question of trust: can we build an evidence base to gain trust in systematic review automation technologies? (2019) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gene Expression Omnibus: NCBI gene expression and hybridization array data repository
Edgar, R. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.1.207 · Cited by 13,288 · cited 1×
“Various solutions have now become available to make data sharing more convenient, standardized, and accessible in particular domains, such as for gene expression data [21– 23].”
BackgroundMinimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109 · cited 1×
“Various solutions have now become available to make data sharing more convenient, standardized, and accessible in particular domains, such as for gene expression data [21– 23].”
BackgroundGoecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome Biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,453 · cited 1×
“For an even tighter integration, there are tools available to help integrate reproducible analyses directly into textual documents, such as Sweave [19], the GenePattern Word add-in [4], and Galaxy Pages [20].”
MethodSelf-citationRaise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“The importance of replication and reproducibility has recently been exemplified through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], a…”
BackgroundReich, M., Liefeld, T., Gould, J., Lerner, J., Tamayo, P., & Mesirov, J. P. (2006) · Nature Genetics · doi:10.1038/ng0506-500 · Cited by 2,032
Galaxy: A platform for interactive large-scale genome analysis
Giardine, B., Riemer, C., Hardison, R. C., Burhans, R., Elnitski, L., Shah, P., Zhang, Y., Blankenberg, D., Albert, I., Taylor, J., et al. (2005) · Genome Research · doi:10.1101/gr.4086505 · Cited by 2,026 · cited 2×
“If you are instead performing your analyses through an integrated framework (such as Gene-Pattern [10], Galaxy [11], LONI pipeline [12], or Taverna [13]), the system may already provide full or partial support for most of the rules.”
BackgroundSelf-citationBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“…through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], and through a high number of failing clinical trials [8,9].”
ResultTaverna: a tool for the composition and enactment of bioinformatics workflows
Oinn, T., Addis, M., Ferris, J., Marvin, D., Senger, M., Greenwood, M., Carver, T., Glover, K., Pocock, M. R., Wipat, A., et al. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth361 · Cited by 1,602 · cited 2×
“Such executable descriptions [10] might come in the form of simple shell scripts or makefiles [15,16] at the command line, or in the form of stored workflows in a workflow management system [10,11,13,17,18].”
BackgroundReproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · cited 3×
“In order to foster such aspects, it has been suggested that the scientific community needs to develop a ‘‘culture of reproducibility’’ for computational science, and to require it for published claims [3].”
BackgroundPegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems
Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G. B., Good, J., et al. (2005) · Scientific Programming · doi:10.1155/2005/128026 · Cited by 1,222 · cited 1×
“Such executable descriptions [10] might come in the form of simple shell scripts or makefiles [15,16] at the command line, or in the form of stored workflows in a workflow management system [10,11,13,17,18].”
Background
Cited by
Items 601–700 of 1,001
A policy change at JAMS: The role of reproducibility in science
Sarstedt, M., Noble, C. H., & Noble, S. M. (2026) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-026-01190-0 · Cited by 2
“Analyses may rely on preprocessing steps and model parameterizations that are not fully documented (Sandve et al., 2013).”
Towards a new paradigm of scientific discovery with socialized artificial intelligencePreprint
Yao, X., Xu, X., Gao, X., Guo, Z., Yang, K., Zhao, D., Zhao, S., Fan, Z., Dong, Y., Li, X., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.02775 · Cited by 2
“…bundle with the approved protocol; code and environment versions; data identity, split, and preprocessing; seeds and configurations; complete logs and raw outputs; parsing and statistical scripts; editable figure and table sources; and links to claims that use or contradict the result [73, 79].”
Mohr, S. (2026) · doi:10.5194/egusphere-2026-5952 · Cited by 2
Kyritsis, K., Pechlivanis, N., & Psomopoulos, F. E. (2023) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2023.1275593 · Cited by 1
“…retracted articles and failing clinical trials calls for the establishment of a minimum set of standards required to back scienti fi c claims (Sandve et al., 2013), with several systematic efforts being undertaken towards this goal, such as FAIR Principles (Wilkinson et al., 2016; Goble et…”
BackgroundMethodVázquez, J. A. P. (2023) · región y sociedad · doi:10.22198/rys2023/35/1805 · Cited by 1
Mandala: Compositional Memoization for Simple & Powerful Scientific Data Management
Makelov, A. (2024) · Proceedings of the Python in Science Conferences · doi:10.25080/jhpv7385 · Cited by 1
“Rules and best practices that help with these requirements exist and are well-known (Sandve et al., 2013; Wilkinson et al., 2016), but still require manual effort, attention to extraneous details, and discipline to follow.”
BackgroundAn open source statistical web application for validation and analysis of virtual cohorts
Ohmann, C., Khorchani, T., Cracanel, A., Brüning, J., & Verde, P. E. (2025) · Scientific Reports · doi:10.1038/s41598-025-99720-3 · Cited by 1
“That there are still issues with reproducibility/replicability has been demonstrated in several publications (e.g. 37 ).”
The Observed Availability of Data and Code in Earth Science and Artificial Intelligence
Jones, E. A., McClung, B., Fawad, H., & McGovern, A. (2025) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-24-0147.1 · Cited by 1
Social Computational Literacy in Practice: A Framework Describing STEM Researchers’ Communication
Cammarota, C. M., Foster, M. S., Verostek, M., Patterson, K., MacIntyre, M., Dorsey, K., Camacho-Betancourt, A. N., Wong, T. E., & Zwickl, B. M. (2025) · Journal for STEM Education Research · doi:10.1007/s41979-025-00155-2 · Cited by 1
Owolabi, O. S. (2025) · Journal of Data Analysis and Information Processing · doi:10.4236/jdaip.2025.133022 · Cited by 3
Brachmann, M. K., & Jacobs, C. T. (2025) · Evolutionary Biology · doi:10.1007/s11692-025-09655-w · Cited by 3
Mapping SET1B chromatin interactions with DamID using DamMapper, a comprehensive Snakemake workflow
Wit, N., Bertlin, J. A. C., Hynes-Allen, A. M., van den Ameele, J., & Nathan, J. A. (2025) · BMC Genomics · doi:10.1186/s12864-025-12075-x · Cited by 1
“Furthermore, our workflow incorporates best practices for reproducible computational research [23], including: • Biological Replicates: Seamlessly handles multiple biological replicates, improving statistical power and allowing for robust identification of binding sites.”
GeneDrive.jl: A decision tool to optimize biological vector control strategies under climate change
Vásquez, V., Mordecai, E. A., & Anthoff, D. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013600 · Cited by 1
Coutray, K., & Constantinidis, C. (2025) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2025.110647 · Cited by 1
“Few existing solutions offer the automatic archival of exact code, configuration files, and visual records that are crucial for robust data provenance (Sandve et al., 2013).”
Learning-Based Multi-Objective Optimization of Parametric Stadium-Type Tiered-Seating Configurations
Kurumak, M., & Bademci, F. (2026) · Mathematics · doi:10.3390/math14030410 · Cited by 1
Zhang, L., Huang, D., Yu, W., Yang, W., Feng, D., Xie, H., & Zhang, H. (2026) · Journal of Pipeline Science and Engineering · doi:10.1016/j.jpse.2026.100473 · Cited by 1
projectLSA : A Shiny Application for Integrated Latent Structure Analysis
Djidu, H., Retnawati, H., Hadi, S., & Haryanto (2026) · Applied Psychological Measurement · doi:10.1177/01466216261446305 · Cited by 1
“While these tools are methodologically robust, transitioning between them requires substantial recon fi guration of work fl ows, increases cognitive load, and can reduce reproducibility due to inconsistencies in model speci fi cation and output structures (Peng, 2011; Sandve et al., 2013).”
Arcos-Argudo, M., Bojorque, R., Pesántez, F., & Nieto-Andrade, K. (2026) · Data · doi:10.3390/data11050103 · Cited by 1
“Reproducibility is increasingly recognized as a fundamental requirement for reliable computational research [5,6].”
InfluentialLiu, L., Kötter, T., Franklin, R., & Wang, F. (2026) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2026.107013 · Cited by 1
Sirago, G., Solarino, B., Dell'Erba, A., & Ferorelli, D. (2026) · International Journal of Legal Medicine · doi:10.1007/s00414-026-03849-3 · Cited by 1
“These practices are well established in computational research and should be adapted to evidence-producing pipelines [85 –88].”
InfluentialOvercoming barriers to computational reproducibility
Hornung, R., Németh, L., Zadorozhnyi, O., Ullmann, T., Kammer, M., Killick, R., Paciorek, C., Chiquet, J., Herrmann, M., Batinović, L., et al. (2026) · Royal Society Open Science · doi:10.1098/rsos.252489 · Cited by 2
Cobb, M., & COGHLAN, S. (2026) · Frontiers in Animal Science · doi:10.3389/fanim.2026.1868726 · Cited by 1
Data Management and Digital Infrastructure in Social Sciences
Sułkowski, Ł., Woźniak, A., Seliga, R., & Lis, M. (2026) · Routledge eBooks · doi:10.4324/9781003790334 · Cited by 1
Eaves, J. N., Needs, A. A., & Woldring, D. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c01192 · Cited by 1
“73,74 Conversely, as we show here, pipelines that satisfy only the minimum checklist are vulnerable to the failure modes documented in this study.”
Ten quick tips to SNIFF out sustainable and secure scientific software
Nagraj, V., Siller, K. H., Stewart, T., Magee, N., & Turner, S. D. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014510 · Cited by 1
“For example, there are Ten Simple Rules for reproducible computational research [8], writing clean and usable code [9], for making research software more robust and portable [10], for writing Dockerfiles to enable reproducible analyses [2], and for making computing practices more environmentally…”
Quang, N. T. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01793-0 · Cited by 1
“Explicit workflow documentation is consistent with the broader principle that empirical computational research should provide enough procedural detail to allow independent understanding and, where feasible, reproduction of the analysis [41, 46].”
Salla, R. L., Henning, E., Nierwinski, H. P., dos Santos, A. G., Silvani, C., Silva, A. A. A. D. (., & Hastenpflug, D. (2026) · Eng—Advances in Engineering · doi:10.3390/eng7080374 · Cited by 1
Bayesian Priors Prediction in Ze
Tqemaladze, J. (2025) · Longevity Horizon · doi:10.65649/931348ba · Cited by 1
Bailo, D., Lauciani, V., Stefano, R. D., & Bernardi, F. (2026) · Earth Science Informatics · doi:10.1007/s12145-026-02197-5 · Cited by 1
“As for traceability and provenance , scientific claims require verifiable evidential support and reproducibility (Sandve et al. 2013).”
A New Paradigm: Agentic AI for Scientific Discovery
Taktakidze, A. (2026) · Longevity Horizon · doi:10.65649/ukh4n33x · Cited by 1
Kang, L., Xinlong, L., Zhou, L., Xingxin, Z. H. A. O., & Wei, J. (2026) · Computers & Geosciences · doi:10.1016/j.cageo.2026.106262 · Cited by 1
Louail, P., Graeve, M. D., Mahmoud, A., Silva, D. M. D. S. E., Nishida, K., Suksi, V., Tagliaferri, A., Tomè, G., Hernandes, V. V., & Rainer, J. (2026) · Metabolomics · doi:10.1007/s11306-026-02538-x · Cited by 1
“Reviews of reproducible and open computational practice have repeatedly made the case that such end-to-end, executable examples are what let readers actually reuse an analysis rather than merely read about it (Grüning et al., 2018 ; Sandve et al., 2013).”
Evaluating FoldX 5.1 for MAVISp stability data collection
Vliora, A., Tiberti, M., & Papaleo, E. (2026) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2026.109624 · Cited by 1
Resilience within Ultrascale Computing System: Challenges and Opportunities from Nesus Project
Bouvry, P., Mayer, R. L., Muszyński, J., Petcu, D., Rauber, A., Tempesti, G., Trinh, T. A., & Varrette, S. (2015) · Supercomputing Frontiers and Innovations · doi:10.14529/jsfi150203 · Cited by 1
“Recognizing these needs, steps are being taken to assist research in ensuring their results are more easily reproducible [24, 25].”
BackgroundApplication of an AIS to the problem of through life health management of remotely piloted aircraft
Pelham, J., Fan, I., Jennions, I. K., & McFeat, J. (2015) · AIAA Infotech@Aerospace Conference · doi:10.2514/6.2015-1797 · Cited by 1
“This also hampers the ability of researchers to do reproducible research to a gold standard.[41] Comparison between different AIS methods is needed for the field to really improve.”
BackgroundHasham, K., Munir, K., McClatchey, R., & Shamdasani, J. (2016) · Communications in computer and information science · doi:10.1007/978-3-319-29582-4_5 · Cited by 1
“[26] also suggested archiving the exact versions of all processes and enabling version control on all scripts used in an experiment.”
BackgroundEvolutionary Scientific Workflows
da Cruz, S. M. S., de Oliveira, A., & de Faria, F. F. (2018) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2018.8477850 · Cited by 1
“The facilities offered by scientific workflows include rapid and straightforward design, reuse and documentation, scalable execution, provenance collection, sharing and collaboration, enhanced reproducibility and other benefits that altogether enable computational research and increase the reproducibility of the exper…”
Stöver, B. C., Wiechers, S., & Müller, K. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2982-3 · Cited by 1
Ribeiro, C. J. S., Santos, L. O. B. D. S., & Moreira, J. (2020) · Informação & Informação · doi:10.5433/1981-8920.2020v25n3p28 · Cited by 1
I want to work in bioinformatics. How do I start?
Franco, C., Maidana, R. R., Rotela, A. T., Noto, M., Arrúa, A. A., & Ríos, D. F. (2020) · Reportes científicos de la FACEN · doi:10.18004/rcfacen.2020.11.1.51 · Cited by 1
Easyreporting simplifies the implementation of Reproducible Research layers in R software
Righelli, D., & Angelini, C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244122 · Cited by 1
“Finally, thanks to its versatility, easyreporting can be ideally included in any well-structured R project and the development of GUIs, helping to fulfill most of the proposed rules in [2].”
BackgroundResultBasic Principles of Bioinformatics for Next-Generation Sequencing Molecular Testing in Oncology
Summa, S. D., & Tommasi, S. (2021) · UNIPA Springer series · doi:10.1007/978-3-030-56051-5_17 · Cited by 1
Capturing and Semantically Describing Provenance to Tell the Story of R Scripts
Mondelli, M. L., Samuel, S., König‐Ries, B., & Gadelha, L. M. R., Jr. (2021) · IEEE International Conference on eScience · doi:10.1109/escience51609.2021.00057 · Cited by 1
“In this sense, some studies have pointed out good practices for the development of reproducible research [7], [8].”
BackgroundDesign of a Scientific Data Analysis Support Platform
Martindale, N., Hite, J. M., STEWART, S. L., & Adams, M. (2022) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-212e5952-01b · Cited by 2
Reproducing a decision-making network in a virtual visual discrimination task
Trapani, A. M., Sheiban, F. J., Bertone, E., Chiosso, S., Colombo, L., D'Andrea, M., Santis, F. D., Fati, F., Fossati, V., Gonzalez, V., et al. (2022) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2022.930326 · Cited by 1
“The concept of reproducibility has indeed witnessed a growing interest among the scientific community over recent years, leading to the proposal of 10 standard guidelines for reproducible computational research (Sandve et al., 2013).”
BackgroundCancelled: Unified Software Design Patterns for Simulated Annealing
Goswami, R. R., Ruhila, S., Goswami, A., Goswami, S., & Goswami, D. (2023) · IntechOpen eBooks · doi:10.5772/intechopen.110675 · Cited by 1
A new powerful tool to easily encode the geo-spatial dimension in population genetics simulations
Trucchi, E. (2023) · Peer Community in Evolutionary Biology · doi:10.24072/pci.evolbiol.100630 · Cited by 1
Workflows for Bioinformatics Data Integration
Cohen‐Boulakia, S., & LEMOINE, F. (2023) · Wiley eBooks · doi:10.1002/9781394257317.ch3 · Cited by 1
Mandl, M. M., Hoffmann, S., Bieringer, S., Jacob, A. E., Kraft, M., Lemster, S., & Boulesteix, A. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011936 · Cited by 1
“Going beyond selective reporting, the course also provides the opportunity to teach students reproducible research practices [23] and to discuss important issues in the design and analysis of experimental studies, including lack of statistical power, nonadherence, and the common misinterpretation…”
BackgroundMirończuk, M., Müller, A., & Pedrycz, W. (2024) · IEEE Access · doi:10.1109/access.2024.3513550 · Cited by 3
“…are crucial questions that should be addressed using a reliable and properly selected study design and validation process to estimate the performance of the proposed solution, and compare it with other relevant solutions [57], [115], [116], [117], [118], [119], [120], [121], [122], [123], [124].”
RCloud – Collaborative Visualization and Analysis Platform
Urbanek, S. (2024) · Journal of Data Science · doi:10.6339/24-jds1153 · Cited by 1
Twelve Tips for Reproducible Analysis of Single-Cell Transcriptomics Data
Onfroy, A., Topilko, P., Hüe, S., & Thieffry, D. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746138 · Cited by 1
“Various computational tools can be used to foster the reproducibility of computational analyses [16, 32, 54, 61, 75, 84].”
Executable Science: Research Software Engineering Practices for Replicating Neuroscience Findings
Palacios, G. K., Li, Z., & Armeni, K. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746147 · Cited by 1
Cáceres-Tello, J., Galán-Hernández, J. J., Cevallos, M. B. M., & Menéses, E. L. (2025) · Applied Sciences · doi:10.3390/app152212183 · Cited by 3
Ringuette, R. (2024) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.10934204 · Cited by 1
Ten simple rules for managing high-throughput nucleotide sequencing dataPreprint
Vos, R. A. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/049338 · Cited by 1
PyBayenv: A framework for interpreting, testing and optimizing Bayenv analyses
Ring, K. H. (2015) · NORA - Norwegian Open Research Archives · Cited by 3
Melchert, O., Blumenröther, E., Meinhardt‐Wollweber, M., & Roth, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05054 · Cited by 1
“Since the non-availability of code impedes transparency and reproducability of results in scientific publications [33, 34, 35] we considered it useful to make the concise research-code for the presented study, including but not limited to the code listings 1 through 6 along with all scripts needed to reproduce all fig…”
Koala: sistema para integração de métodos de predição e análise de estruturas de proteína
Defelicibus, A. (2016) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.82.2016.tde-22062016-102823 · Cited by 1
Möller, S. (2016) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000184546 · Cited by 1
“’s “Ten Simple Rules for Reproducible Computational Research” [76] and Noble’s article on how to organize Computational Biology projects [77].”
BackgroundComputational Pipelines and Workflows in Bioinformatics
Leipzig, J. N. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809633-8.20187-8 · Cited by 3
Reproducible data citations for computational researchPreprint
Schulz, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07541 · Cited by 1
“It has been suggested that reproducibility should be a minimum standard in the computational sciences [3, 4].”
BackgroundArchitecture de dataflow pour des systèmes modulaires et génériques de simulation de plante
Pradal, C. (2019) · theses.fr (ABES) · doi:10.70675/2e4d7f3dz47e8z42d5zbf84z18f773d59a0a · Cited by 2
Jimenez, I. (2019) · eScholarship (California Digital Library) · Cited by 1
Gonçalves, A. J., Alotaibi, M., Carlin, A., Chen, Y., Wong, C., Djoukhadar, I., Krokos, G., Jackson, A., Coope, D. J., & Asselin, M. (2021) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x211061050 · Cited by 1
congressbr: An R Package for Analysing Data from Brazil's Chamber of Deputies and Federal SenatePreprint
McDonnell, R. M., Duarte, G., & Freire, D. (2017) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/n5jd8 · Cited by 2
Computational Social Welfare: Applying Data Science in Social Work
Ren, C., Stuart, M. A., & Chow, J. C. (2020) · Oxford University Press eBooks · doi:10.1093/obo/9780195389678-0286 · Cited by 3
Publishing Reproducible Geoscientific Papers: Status quo, benefits, and opportunitiesPreprint
Konkol, M. (2019) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/mcdrn · Cited by 1
An intuitive guide to processing RNA-seq readsPreprint
Prodhan, M. A. (2023) · Research Square · doi:10.21203/rs.3.rs-2806717/v1 · Cited by 1
Kaur, R. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95502-7.00135-4 · Cited by 1
pipemake: A pipeline creation tool using Snakemake for reproducible analysis of biological datasetsPreprint
Webb, A. E., Wolf, S., Traniello, I. M., & Kocher, S. D. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.20.629758 · Cited by 1
Torre, D. L., & Chicco, D. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2986 · Cited by 1
“To comply with the reproducibility principles of open science ( Sandve et al., 2013 ; Iqbal et al., 2016 ), we report here the instructions to reproduce the results described in this study from any personal computer having Python installed and an internet connection.”
Taylor, J. S., Shoppell, E. A., Bargmann, B. O. R., & Wright, R. C. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.26.666912 · Cited by 1
“However, reproducibility of data analysis is crucial (Sandve et al. 2013).”
Self-citationSchlögl, M., Waltersdorfer, L., Regner, P., Siposova, A., & Brenning, A. (2025) · doi:10.5194/egusphere-2025-5210 · Cited by 1
Schlögl, M., Waltersdorfer, L., Regner, P., Siposova, A., & Brenning, A. (2025) · doi:10.5194/egusphere-2025-5210-v1 · Cited by 1
Wegener, G. H. (2025) · SSRN Electronic Journal · doi:10.20944/preprints202511.1783.v3 · Cited by 1
Batch Effect Correction in a Functional Colorectal Cancer Organoid Clinical Correlation StudyPreprint
Oliver, G. R., Domingues, A. M. D. J., & Barnett, C. C. (2026) · Preprints.org · doi:10.64898/2026.02.05.704065 · Cited by 1
Information-Theoretic Methods in Spatial TranscriptomicsPreprint
Fuentes, V. G. H., Tan, M. J. T., Kapetanaki, M. G., Rashidi, P., Huang, K., & Benos, P. V. (2026) · doi:10.36227/techrxiv.177155947.75111980/v1 · Cited by 1
Zhu, Y., Lionts, M., Haugen, E. J., Walter, A. B., Voss, T., Grow, G., Liao, R.-Y., McKee, M. E., Ahmed, R., Afreh, J. K., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.26.714582 · Cited by 1
“18 Although these principles are widely recognized, practical tools that operationalize re-producibility in Raman spectral processing remain limited.”
fluorojip: Analysis of Chlorophyll a Fluorescence Transient Parameters
Ribeiro, J. E. D. S., da Silva, T. I., & Rodriguez, R. M. (2026) · doi:10.32614/cran.package.fluorojip · Cited by 1
bioLeak: Leakage-Aware Modeling and Diagnostics for Machine Learning in RPreprint
Korkmaz, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.10965 · Cited by 1
“This design is consistent with the principle that computational claims should remain inspectable at the level of code, data products, and derived summaries [Gentleman and Lang, 2007, Peng, 2011, Sandve et al., 2013].”
OmicOS: A Comprehensive Omics Ecosystem Infrastructure and Agent System for the AI EraPreprint
Zeng, Z., Meng, X., Hu, L., Li, C., Liu, P., Shi, Y., Ma, X., Gao, L., Wang, X., Luo, Z., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.11.731775 · Cited by 1
Ba, K., Thiébaut, R., Hinaut, X., & Hejblum, B. P. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.17.732876 · Cited by 1
“Beyond performance, reproducibility and methodological transparency are essential requirements for computational tools intended for broad scientific use [24, 26].”
BlotTool: A reproducible command-line workflow for western blot densitometry
Sanders, M. T. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102859 · Cited by 1
SciForge: An AI-Native, Multimodal Workbench for Scientific DiscoveryPreprint
SciForge Team, Gao, Z., Fang, M., Liu, Y., Hj, Y., Gu, X., Sk, T., Sun, S. Y., Bai, L., Tan, C., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16038 · Cited by 1
“Auditable evidence traces must link every conclusion to its supporting data, script, model, parameter, paper, or expert translation, and flag when figures or actions require human-PI approval; these requirements align with established FAIR and reproducible-computation principles [52, 61].”
蒋, 涵., Gao, F., Liu, P., Wu, Y., Jie, Y., Li, Y., & Jiang, Y. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.10.750775 · Cited by 1
Designing reproducible large-language-model-assisted scientific analyses
Dunn, C. W., Schultz, D. T., & Musser, J. M. (2026) · Patterns · doi:10.1016/j.patter.2026.101644 · Cited by 1
“2,3,6 The rough consensus reduces to three ingredients.”
A Model To Structure Scientific Information For Decision Makers
Schmidt, R., Barrett, D. J., & Gallant, S. (2014) · CUNY Academic Works (City University of New York)
Irving, D. (2016) · Minerva Access (University of Melbourne)
Reproducibility and reusability of genome assembly evaluation
Ifzal, S. (2014) · NORA - Norwegian Open Research Archives
Low Template, Low Level or Low Copy Number DNA
Taupin, J. M. (2018) · CRC Press eBooks · doi:10.4324/9781315381930-6
Reproducible Database Queries in Privacy Sensitive Applications
Pröll, S., Mayer, R. L., & Rauber, A. (2015) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2015.05.202
Pradal, C., Artzet, S., Chopard, J., Dupuis, D., Fournier, C., Mielewczik, M., Nègre, V., Neveu, P., Parigot, D., & Valduriez, P. (2017)
Having your cake and eating it too: Scripted workflows for image manipulationPreprint
Thompson, P. A., Matloff, N., Fu, A., & Shin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07406
“Again, this must be carefully documented to produce valid outcomes ([31] [5] [38] [43]).”
BackgroundMethodAlgorithms for Reconstruction of Undersampled Atomic Force Microscopy Images
Oxvig, C. S. (2017) · VBN Forskningsportal (Aalborg Universitet) · doi:10.5278/vbn.phd.engsci.00158
Pipeline of High Throughput Sequencing
Ong, C. K., Kwong, Q. B., Ong, A. L., Heng, H. Y., & Low, W. Y. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-809633-8.20091-5
Constructing Computational Pipelines
Ong, C. K., & Hamilton, R. S. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-809633-8.20090-3
Evaluating Reproducibility in Computational Biology Research
Oneka, M. D. (2018) · Language arts journal of Michigan
Metadata
- Authors
- Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E.
- Affiliations
- University of Oslo (NO)
- Pennsylvania State University (US)
- Emory University (US)
- Oslo University Hospital (NO)
- Corresponding author
- Sandve, Geir Kjetil
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-10-24
- Volume
- 9
- Issue
- 10
- Pages
- e1003285
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3812051
- OpenAlex
- W2036318837
- Cited by
- 880
- References
- 27
- Citations by year
- 2026: 176 · 2025: 60 · 2024: 52 · 2023: 50 · 2022: 58 · 2021: 67 · 2020: 63 · 2019: 88 · 2018: 76 · 2017: 64 · 2016: 56 · 2015: 42 · 2014: 24 · 2013: 3
- FWCI
- 91.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (0850103)
- Pennsylvania Department of Health (HG004909)
- University of Pennsylvania
- Pennsylvania State University
- National Institutes of Health (HG006620)
- Huck Institutes of the Life Sciences
- Institute for Computational and Data Sciences, Pennsylvania State University
- National Human Genome Research Institute (HG005133, U41HG006620, R21HG005133, R01HG004909, HG006620, HG004909)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- reproducible research, computational reproducibility, research practices, bioinformatics analysis, research transparency, version control, software dependency management, workflow management, research data management, research culture, research evaluation, research dissemination, scholarly communication
- MeSH terms
- Reproducibility of Results, Computational Biology, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.