Ten Simple Rules for Reproducible Computational Research
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Replication is the cornerstone of a cumulative science [1]. However, new tools and technologies, massive amounts of data, interdisciplinary approaches, and the complexity of the questions being asked are complicating replication efforts, as are increased pressures on scientists to advance their research [2]. As full replication of studies on independently collected data is often not feasible, there has recently been a call for reproducible research as an attainable minimum standard for assessing the value of scientific claims [3]. This requires that papers in experimental science describe the results and provide a sufficiently clear protocol to allow successful repetition and extension of analyses based on original data [4]. The importance of replication and reproducibility has recently been exemplified through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], and through a high number of failing clinical trials [8], [9]. This has led to discussions on how individual researchers, institutions, funding bodies, and journals can establish routines that increase transparency and reproducibility. In order to foster such aspects, it has been suggested that the scientific community needs to develop a “culture of reproducibility” for computational science, and to require it for published claims [3]. We want to emphasize that reproducibility is not only a moral responsibility with respect to the scientific field, but that a lack of reproducibility can also be a burden for you as an individual researcher. As an example, a good practice of reproducibility is necessary in order to allow previously developed methodology to be effectively applied on new data, or to allow reuse of code and results for new projects. In other words, good habits of reproducibility may actually turn out to be a time-saver in the longer run. We further note that reproducibility is just as much about the habits that ensure reproducible research as the technologies that can make these processes efficient and realistic. Each of the following ten rules captures a specific aspect of reproducibility, and discusses what is needed in terms of information handling and tracking of procedures. If you are taking a bare-bones approach to bioinformatics analysis, i.e., running various custom scripts from the command line, you will probably need to handle each rule explicitly. If you are instead performing your analyses through an integrated framework (such as GenePattern [10], Galaxy [11], LONI pipeline [12], or Taverna [13]), the system may already provide full or partial support for most of the rules. What is needed on your part is then merely the knowledge of how to exploit these existing possibilities. In a pragmatic setting, with publication pressure and deadlines, one may face the need to make a trade-off between the ideals of reproducibility and the need to get the research out while it is still relevant. This trade-off becomes more important when considering that a large part of the analyses being tried out never end up yielding any results. However, frequently one will, with the wisdom of hindsight, contemplate the missed opportunity to ensure reproducibility, as it may already be too late to take the necessary notes from memory (or at least much more difficult than to do it while underway). We believe that the rewards of reproducibility will compensate for the risk of having spent valuable time developing an annotated catalog of analyses that turned out as blind alleys. As a minimal requirement, you should at least be able to reproduce the results yourself. This would satisfy the most basic requirements of sound research, allowing any substantial future questioning of the research to be met with a precise explanation. Although it may sound like a very weak requirement, even this level of reproducibility will often require a certain level of care in order to be met. There will for a given analysis be an exponential number of possible combinations of software versions, parameter values, pre-processing steps, and so on, meaning that a failure to take notes may make exact reproduction essentially impossible. With this basic level of reproducibility in place, there is much more that can be wished for. An obvious extension is to go from a level where you can reproduce results in case of a critical situation to a level where you can practically and routinely reuse your previous work and increase your productivity. A second extension is to ensure that peers have a practical possibility of reproducing your results, which can lead to increased trust in, interest for, and citations of your work [6], [14]. We here present ten simple rules for reproducibility of computational research. These rules can be at your disposal for whenever you want to make your researc
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rough consensus reduces to three ingredients.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While there have been a number of recent, often domain-focused publications advocating for specific improvements in practices relating to data management and archival1,11,12, FAIR differs in that it describes concise, domain-independent, high-level principles that can be applied to a wide range of scholarly outputs.”
From The FAIR Guiding Principles for scientific data management and stewardship (2016) · cites this paper as [12]
In the citing paper
“In a recent publication reviewing the automated citation screening methods reported for systematic reviews, the number of studies that met the current standards for reproducible software engineering experiments was low [29, 30, 36]; of 33 studies reporting approaches to automated screening, no study provided all the criteria for reproducible reporting [36].”
From A question of trust: can we build an evidence base to gain trust in systematic review automation technologies? (2019) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gene Expression Omnibus: NCBI gene expression and hybridization array data repository
Edgar, R. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.1.207 · Cited by 13,288 · cited 1×
“Various solutions have now become available to make data sharing more convenient, standardized, and accessible in particular domains, such as for gene expression data [21– 23].”
BackgroundMinimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109 · cited 1×
“Various solutions have now become available to make data sharing more convenient, standardized, and accessible in particular domains, such as for gene expression data [21– 23].”
BackgroundGoecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome Biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,453 · cited 1×
“For an even tighter integration, there are tools available to help integrate reproducible analyses directly into textual documents, such as Sweave [19], the GenePattern Word add-in [4], and Galaxy Pages [20].”
MethodSelf-citationRaise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“The importance of replication and reproducibility has recently been exemplified through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], a…”
BackgroundReich, M., Liefeld, T., Gould, J., Lerner, J., Tamayo, P., & Mesirov, J. P. (2006) · Nature Genetics · doi:10.1038/ng0506-500 · Cited by 2,032
Galaxy: A platform for interactive large-scale genome analysis
Giardine, B., Riemer, C., Hardison, R. C., Burhans, R., Elnitski, L., Shah, P., Zhang, Y., Blankenberg, D., Albert, I., Taylor, J., et al. (2005) · Genome Research · doi:10.1101/gr.4086505 · Cited by 2,026 · cited 2×
“If you are instead performing your analyses through an integrated framework (such as Gene-Pattern [10], Galaxy [11], LONI pipeline [12], or Taverna [13]), the system may already provide full or partial support for most of the rules.”
BackgroundSelf-citationBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“…through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], and through a high number of failing clinical trials [8,9].”
ResultTaverna: a tool for the composition and enactment of bioinformatics workflows
Oinn, T., Addis, M., Ferris, J., Marvin, D., Senger, M., Greenwood, M., Carver, T., Glover, K., Pocock, M. R., Wipat, A., et al. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth361 · Cited by 1,602 · cited 2×
“Such executable descriptions [10] might come in the form of simple shell scripts or makefiles [15,16] at the command line, or in the form of stored workflows in a workflow management system [10,11,13,17,18].”
BackgroundReproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · cited 3×
“In order to foster such aspects, it has been suggested that the scientific community needs to develop a ‘‘culture of reproducibility’’ for computational science, and to require it for published claims [3].”
BackgroundPegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems
Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G. B., Good, J., et al. (2005) · Scientific Programming · doi:10.1155/2005/128026 · Cited by 1,222 · cited 1×
“Such executable descriptions [10] might come in the form of simple shell scripts or makefiles [15,16] at the command line, or in the form of stored workflows in a workflow management system [10,11,13,17,18].”
Background
Cited by
Items 901–1,000 of 1,001
The HERMES Framework for Experimental Reproducibility
Oga, E., Yolandia, J. N. E., & Mughe, H. C. (2026) · Laboratories · doi:10.3390/laboratories3030017
“Foundational implementation emphasizes metadata integrity, contextual documentation, environmental awareness, contamination monitoring, calibration verification, preprocessing transparency, and personnel training [37,68].”
InfluentialSims, H. C., Sims, J. J., Guirgus, M. M., Grames, C., Simental, A., & Khan, S. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15197636
“…spatial-transcriptomic data [44], community best-practice guidelines for single-cell and spatial analysis [45,49], guidance on batch effects in high-throughput data [46], the FAIR principles for data and code availability [47], and recommendations for reproducible computational research [48].”
Muthumaniraja, S. (2026) · ShodhAI Journal of Artificial Intelligence · doi:10.29121/shodhai.v3.i2.2026.97
Flores, F. D. L. C., Flores, P. A. D. L. C., Garcia, T., & Frausto, H. R. (2026) · Journal of Integrative Science and Societal Impact · doi:10.29121/jissi.v2.i2.2026.35
Évaluation environnementale des conséquences de la décarbonisation des services énergétiques
Astudillo, M. F. (2019) · Knowledge UdeS (Institutional Deposit of the University of Sherbrooke)
YU, C., Elias, N. F., Yahya, Y., Jenal, R., & Fan, M. (2026) · Forecasting · doi:10.3390/forecast8040068
“The archive follows the principle that code, data, parameters, and computational steps should be retained together to permit reproducible verification [54,55].”
Ndahawali, E., Ardika, I. K., Priyono, Pangow, T. T. V., & Warroka, A. (2026) · EDUCATIONE · doi:10.59397/edu.v4i2.288
“These disclosures follow established expectations for reproducible computational research, including accessible code, complete inputs, explicit parameters, and traceable analysis steps (Sandve et al., 2013).”
GEOMETRY USED AS A TOOL FOR DESIGN CONCEPTION
de Lima, M. M. F. (2026) · Brazilian Journals Editora eBooks · doi:10.35587/brj.ed.978-65-6016-158-0_6
do Nascimento, E. S., de Araújo, K. F. G., Mashhadikarimi, M., & Gomes, U. U. (2026) · Brazilian Journals Editora eBooks · doi:10.35587/brj.ed.978-65-6016-158-0_7
Cuartero, C. T., Lambert, J. W., Cuomo-Dannenburg, G., Fitzner, J., Ndumbi, P., Rice, C., Saulo, D., Waddell, L. A., Azam, J. M., Doohan, P., et al. (2026) · Epidemics · doi:10.1016/j.epidem.2026.100942 · Cited by 1
Li, N., Hu, J., Tong, W., Liu, C., Ding, Y., Li, N., & Cai, Z. (2026) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2026.1906036
“The analysis record therefore remains consistent from feature filtering to model evaluation and visualization (Sandve et al., 2013; Barker et al., 2022).”
Reis, D. A. P., Rodrigues, B. C., Takahashi, R. J., Fazan, L. H., Soares, L. F. P., & Silva, L. C. (2026) · Brazilian Journals Editora eBooks · doi:10.35587/brj.ed.978-65-6016-158-0_8
ATTITUDE DYNAMICS OF SPIN STABILIZED SATELLITE: GSAMQ PROPAGATOR
de Oliveira, T. D. M., Zanardi, M. C. F. D. P. S., & Santos, C. C. C. D. P. (2026) · Brazilian Journals Editora eBooks · doi:10.35587/brj.ed.978-65-6016-158-0_9
Cheung, N. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1855963
“To study the joint effects of coupling, lattice geometry, and environmental structure, a minimal open-quantum-systems pipe-line was built in QuTiP (11 – 17).”
Select or discard? A machine learning approach in common bean breeding
Cerutti, P. H., Carbonari, L. T. D. S., Coimbra, J. L. M., de Souza, M. B., & Júnior, C. Z. J. (2026) · Agronomy Journal · doi:10.1002/agj2.70511
Wang, X., Liu, W., & Huang, L.-C. (2026) · Symmetry · doi:10.3390/sym18081402 · Cited by 1
“It also follows principles of reproducible computational research that link reported claims to inspectable observations, explicit transformations, and reusable analytical procedures [40,41].”
Proteus: an executable, statically-checked language for biological reaction modelsPreprint
Deprez, F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7324287
de Faria, V. G., Figueira, J. A., Brandão, R. D. S., Brandão, J. L., Lopes, J. D. S., de Figueirêdo, N. M. A., Porto, F., & da Silva, C. R. L. (2026) · Revista de Pesquisa Cuidado é Fundamental Online · doi:10.9789/2175-5361.rpcfo.v18.15154
CepalStatR: Reproducible access to Latin American official statistics from CEPALSTAT in R
Osorto, H. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102973
Моложон, К. О., & Шило, О. М. (2026) · Scientific Notes of Junior Academy of Sciences of Ukraine · doi:10.51707/2618-0529-2026-36-06
Qiu, H., & Wang, X. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102970
Hernández, A. M., Amaya, S. L. F., & Osorio, C. I. M. (2026) · Research Square · doi:10.21203/rs.3.rs-10707112/v1
CoTRA: a comprehensive R/Shiny framework for transparent bulk and single-cell RNA-seq analysisPreprint
Seemab, U., Vainionpää, K., Tanoli, Z., & Leinonen, H. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.25.747017
EDUCATIONAL FORENSIC METALLURGY AS A FRAMEWORK FOR FAILURE- BASED LEARNING IN ENGINEERING
Campanelli, L. C., Alves, M. V. S., & Reis, D. A. P. (2026) · Brazilian Journals Editora eBooks · doi:10.35587/brj.ed.978-65-6016-158-0_11
Zhao, Y., Jia, Y., Liao, H., Ma, Y., Zhang, Z., Zheng, Y., Zhang, J., Xie, H., & Wang, P. (2026) · Research Square · doi:10.21203/rs.3.rs-10669004/v1
LinkedEarth: a research ecosystem for open and reproducible paleoclimatologyPreprint
Khider, D., Emile‐Geay, J., Ratnakar, V., McKay, N. P., Landers, J. P., Gil, Y., Garijo, D., Edge, D., Routson, C., Gondhalekar, T., et al. (2026) · doi:10.22541/essoar.15008010/v1
Miguel-Medina, T. (2026) · Research Square · doi:10.21203/rs.3.rs-10884006/v1
Filho, R. F. N. (2026) · Qeios · doi:10.32388/05u7a8.2
Rajaei, R., Ghazinoori, S., & Goodarzi, M. S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7382931
Goel, S. (2026) · Asian Bioethics Review · doi:10.1007/s41649-026-00423-2
TEI, Taxonomies, and Semantic Data Governance for Documented LLM-Assisted Literary Analytics
Giménez, D. (2026) · Analytics · doi:10.3390/analytics5030033
“The publication of interpretive datasets also intersects with FAIR-oriented and reproducible research practices [24,25].”
Yao, Q., Pang, J., & Zhang, X. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.26.747436
“Reproducible computational research also requires explicit control of inputs, software, parameters, and execution environments [13, 7, 6, 9, 8].”
Hagen, C., SanClements, M. D., Cross, H., Weintraub, S. R., & Ayres, E. (2026) · Research Square · doi:10.21203/rs.3.rs-10629783/v1
Siqueira, A. M. D. O., Castelo-Grande, T., Mishra, L. N., & da Costa, D. G. (2026) · The Journal of Engineering and Exact Sciences · doi:10.18540/jcecvl12iss1pp24748
Alves, P., Neto, A. S. S., Galdino, M. K. C., da Costa, L. C. C., do Nascimento, D. C., Santos, M., & Minervino, C. A. D. S. M. (2026) · Software Impacts · doi:10.1016/j.simpa.2026.100859
Du, H., Huang, X., Liu, T., & Dolkun, A. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1905644
“…are often completed through a scattered script workflow that may be sufficient for a single analysis but is less convenient for preserving a full decision trail and the associated analytical objects (Abou Hajal and Al Meslamani, 2024; Wheeler et al., 2023; Wilson et al., 2014; Sandve et al., 2013).”
InfluentialCHENG, W. K. W. (2026) · Research Square · doi:10.21203/rs.3.rs-10884877/v1
Roohi, E. (2026) · AI Thermal Fluids · doi:10.1016/j.aitf.2026.100064
Flores, H., Rudolph, T., & Benndorf, J. (2026) · Green and Smart Mining Engineering · doi:10.1016/j.gsme.2026.05.001
Gioanni, N., Prasad, A., Parnell, E. J., Miller, M. P., & Lagesse, B. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7435381
Reduction based dynamical systems analysis of nonlinear wave equations: A review
Saha, N., & Ray, A. (2026) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2026.119397
Clemons, C. A., McKibben, J. D., & Lindner, J. R. (2026) · Journal of Agricultural Education · doi:10.5032/jae.v67i3.3409
Correia, R. C. M., Toledo, D. F., & da Silva, C. T. S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.08184
Amini, A., & Avazpour, S. (2026) · Research Square · doi:10.21203/rs.3.rs-10527737/v1
Javed, Z., Anjum, I. F., & Ansari, A. A. (2026) · Evolutionary Bioinformatics · doi:10.1177/11769343261490077
JSLAB: An Open-Source JavaScript Environment for Scientific Computing and Visualization
Petrašinović, M. D., Raičević, N. G., Petrović, M. G., & Grbović, A. (2026) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-032-33352-0_4
Chen, J., Ying, C., Chen, Y., Zhu, H., & Han, D. (2026) · Applied Sciences · doi:10.3390/app16199420
CHENG, W. K. W. (2026) · Research Square · doi:10.21203/rs.3.rs-10884877/v3
Electroantennography: Less common detection method in gas chromatography
Bílek, T., Hradecký, J., & Švec, F. (2026) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2026.119152
Nunes, S., & Patino-Alonso, M. C. (2026) · Multimodal Technologies and Interaction · doi:10.3390/mti10100100
Aladegba, O. A., & Ayinde, M. (2024) · International Journal of Technology Management and Humanities · doi:10.21590/10.03.15
Audit-aware machine-learning surrogates for endpoint pressure differences in curved pipelinesPreprint
Santos, F. H. D. S., Martins, R. C., Souza, G. D. J., Silva, R. S., Agostinho, L. L., Júnior, E. D. S. F., Silva, P. B., Diogenes, R. S. T. D. M., de Sant’Anna, M. C. S., & de Moura, A. G. L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7511739
Kejriwal, M. (2026) · Proceedings of the 4th ACM Conference on Reproducibility and Replicability · doi:10.1145/3820002.3828595
“Yet a persistent tension exists in computational science: while the importance of code availability and reproducibility is well-established [24, 26, 27], the quality and usability of research software often remain secondary concerns.”
Sinha, A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.23.753765
Building Open Science Into Graduate Training in Clinical Psychology
Girard, J. M. (2026) · Clinical Psychological Science · doi:10.1177/21677026251409757
Bai, S., Meng, Q., Wu, S., Wang, J., Yang, Q., Cui, Y., Ma, S., & Wang, D. (2026) · Genes · doi:10.3390/genes17101199
“Additional strengths include two module-scoring strategies, synthesis restricted to non-discovery datasets, adipose-specific deconvolution, marker-score checks, composition-adjusted sensitivity analyses, and retained analysis code [59,60].”
SHM-EM: a forecast-aware event management framework for heterogeneous engineering monitoring
Liao, J., Wang, Z., Zhao, D., Wang, J., Li, Z., & Yang, S. (2026) · SoftwareX · doi:10.1016/j.softx.2026.103082
Reproducibility in biomedical research
Zhong, J., D’Angelo, D., Lyu, C., Xu, I., Enns, P. K., Wang, F., & Kaushal, R. (2026) · Nature Medicine · doi:10.1038/s41591-026-04667-1
“…transparency requirements 65,74,75,79,89,115,116,125–127 ; enforcement-oriented policies 208 ; AI-specific reporting standards 34,100,174,209,210 Limited editorial and computational verification capacity; lack of in-house technical expertise; concerns about publication delay; costs for…”
Ishiaku, A. (2026) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1109/iccta70480.2026.11706401
“In parallel, reproducibility scholarship has emphasized transparent environments, executable procedures, and reporting discipline in computational science [13], [14], [15], [16], [17], [18].”
Akhtar, M. A. K. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7544499
Hortal, G. C. (2026) · Deposito de Investigacion Universidad de Sevilla (University of Seville)
Sokolov, V., & Skladannyi, P. (2026) · Borys Grinchenko Kyiv University Institutional repository (Borys Grinchenko Kyiv University)
BADESCU, G., SUSINSKI, M., Vasile, C., SĂVESCU, P., Nistor, C., Filip, L. O., Savu, A., & Didulescu, C. (2026) · AI · doi:10.3390/ai7100394
Shamim, A., & Chakraborty, A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.25.754010
“All random seeds were set to 42 throughout the analysis to ensure reproducibility (Sandve et al., 2013).”
A REPRODUCIBLE DATA-ENGINEERING FRAMEWORK FOR MEASURING DIGITAL COMMERCE IN AZERBAIJAN
All Sciences Academy (2026) · International Journal of Advanced Natural Sciences and Engineering Researches · doi:10.5281/zenodo.23102331
Al-Sinjilawi, H. (2026) · Journal of Computer Science and Technology Studies · doi:10.32996/jcsts.2026.8.8.9
Kumbh 2015 Telecom Dataset for Reproducible Mass Gathering Research
Addepalli, L., Shinde, S. K., Pachorkar, S. R., Pagare, G. P., & Lloret, D. J. (2026) · International Journal of Computer Engineering in Research Trends
The Structural Determinants of Low-information Publication in Science: A Constraint-verdict TypologyPreprint
Zhang, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7528578
Brueckner, A., Patel, D., He, Y., & Kassis, T. (2026) · arXiv (Cornell University)
Metriplane: Reproducible incident evidence and presence-based replay checks for bounded workcells
Parkkinen, M. (2026) · SoftwareX · doi:10.1016/j.softx.2026.103104
Пан, А. Н., Argynova, A., Kantarbaeva, D. O., Makhmet, K., Mukanov, Y. B., Sadykov, T. K., Таутаев, Е. М., & Umirzakov, A. (2026) · Applied Sciences · doi:10.3390/app16199816
Corradini, E. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117150
del Castillo, J. G., Bernal, N. C., & Alcocer, I. B. (2026) · Methods in Psychology · doi:10.1016/j.metip.2026.100282
ÍNDICE MEXICANO DE IGUALDAD INTERSECCIONAL (IMI): LECCIONES ONU–UEROPUESTA Y PARA MÉXICO
Sánchez, R. L., & Contreras, N. (2025) · Argumenta Journal Law
AI and the PI: Infinite Generation, Finite Understanding
Giorgino, T. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c03139
Montenegro, M. (. G. (2026) · Repositorio Digital (Escuela Politécnica Nacional)
Cloud-Native Orchestration of Agent-Based Simulations for Energy Systems Research
Schwarz, H., Ma, Z. G., & Jôrgensen, B. N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-41867-8_24
Supercomputer and the Expansion of the Virtual Experiment SpacePreprint
Jaafar, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7492498
GOLIAT: A configurable workflow for near- and far-field RF-EMF dosimetry in Sim4LifePreprint
Wydaeghe, R., Stroobandt, B., Vermeeren, G., Tanghe, E., Guxens, M., & Joseph, W. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7590236
Liu, T., Zhang, H., Jia, Y., & Ning, K. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.30.755547
Bósquez-Barcenes, V. A., Gaibor, J. B. S., Silva, D. Y., & Real, J. V. A. (2026) · doi:10.70577/drxagd55/acacfesa.editorial
Niknam, M., Zohrabzadeh, M., Hahashempour, A., Jahromi, A. H., Karbalaee-Alinazari, M.-M., & Hajizade, M. S. (2026) · Bioinformatics and Biology Insights · doi:10.1177/11779322261489936
AgentEvolver: System-Wide Self-Evolution Through Task ExecutionPreprint
Zhang, W., Yang, F., Zhao, Y., Wang, X., & An, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.11613
Ige, O. O., Olanrewaju, O. A., & Mora‐Camino, F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7590100
gp3sequences: An Auditable and Reproducible Framework for Ordered Categorical Sequence Analysis in R
Balaskas, S. (2026) · AppliedMath · doi:10.3390/appliedmath6100169
Kader, M. A. A., Hernández‐del‐Valle, M., & Harańczyk, M. (2026) · ChemRxiv · doi:10.26434/chemrxiv.15010138/v1
Unuk, G. B. F., & Durak, Z. (2026) · Applied Sciences · doi:10.3390/app16199944
Ma, Z., Zhou, L., & Chen, L. (2026) · Journal of Physics Conference Series · doi:10.1088/1742-6596/3319/1/012023
EthoTrace: from behaviour to neural dynamics in a unified analysis systemPreprint
Oo, H., Tao, C., & Zhang, G. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.04.756578
Publication-date handling policies reshape temporal benchmarks for natural-product target retrievalPreprint
Hu, C., Zhou, X., Guo, Y., & Zhang, X. (2026) · Research Square · doi:10.21203/rs.3.rs-11137779/v1
Hu, Z., & Dillworth, R. (2026) · ChemRxiv · doi:10.26434/chemrxiv.15010253/v1
Allouche, Y. E., Naanaai, L., Marjany, S. E., Ech-Chahdi, Y., Rhabori, S. E., Zaitan, H., & Khalil, F. (2026) · Letters in Drug Design & Discovery · doi:10.1016/j.lddd.2026.100628
Khandaker, M. R., Ghiami‐Shomami, A., & Chowdhury, M. A. (2026) · ChemRxiv · doi:10.26434/chemrxiv.15010372/v1
Eidolon: An Architecture for the Reproducible Publication of Scientific Data Collections
Calmon, L., Junior, M., Santos, L., Alexandrino, F., Coutinho, R., Guedes, G. P., Carvalho, D., & Ogasawara, E. (2026) · doi:10.5753/bresci.2026.249332
“In benchmark settings, it is not enough for data to be available in a repository: they must also be discoverable, documented, and consumable in a way that is integrated with the environment in which experiments are executed [Wilkinson et al., 2016; Peng, 2011; Sandve et al., 2013].”
Task-Scoped Evidence Profiles for External Data Catalog Selection and Traceable Machine Learning
Wang, R., & Wang, X. (2024) · Journal of AI Analytics and Applications · doi:10.63646/itlf6323
“(Sandve et al., 2013).”
Metadata
- Authors
- Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E.
- Affiliations
- University of Oslo (NO)
- Pennsylvania State University (US)
- Emory University (US)
- Oslo University Hospital (NO)
- Corresponding author
- Sandve, Geir Kjetil
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-10-24
- Volume
- 9
- Issue
- 10
- Pages
- e1003285
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3812051
- OpenAlex
- W2036318837
- Cited by
- 880
- References
- 27
- Citations by year
- 2026: 176 · 2025: 60 · 2024: 52 · 2023: 50 · 2022: 58 · 2021: 67 · 2020: 63 · 2019: 88 · 2018: 76 · 2017: 64 · 2016: 56 · 2015: 42 · 2014: 24 · 2013: 3
- FWCI
- 91.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (0850103)
- Pennsylvania Department of Health (HG004909)
- University of Pennsylvania
- Pennsylvania State University
- National Institutes of Health (HG006620)
- Huck Institutes of the Life Sciences
- Institute for Computational and Data Sciences, Pennsylvania State University
- National Human Genome Research Institute (HG005133, U41HG006620, R21HG005133, R01HG004909, HG006620, HG004909)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- reproducible research, computational reproducibility, research practices, bioinformatics analysis, research transparency, version control, software dependency management, workflow management, research data management, research culture, research evaluation, research dissemination, scholarly communication
- MeSH terms
- Reproducibility of Results, Computational Biology, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.