Best Practices for Scientific Computing
Open accessThe DOI matches its Crossref record; 73 of the 75 references it cites were found in a registry or an index.
Abstract
The Community Page is a forum for organizations and societies
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The creation of MIMIC-IV followed best practices in scientific computing23.”
From MIMIC-IV, a freely accessible electronic health record dataset (2023) · cites this paper as [23]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
5 retracted
Chang, G. A., Roth, C., Reyes, C. L., Pornillos, O., Chen, Y.-J., & Chen, A. P. (2006) · Science · doi:10.1126/science.314.5807.1875b · Cited by 137
RetractedRETRACTED: Structure of the multidrug resistance efflux transporter EmrE from Escherichia coli
Ma, C., & Chang, G. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0400137101 · Cited by 102
RetractedChang, G. A. (2007) · Journal of Molecular Biology · doi:10.1016/j.jmb.2003.05.001 · Cited by 22
RetractedMa, C., & Chang, G. A. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0700711104 · Cited by 18
RetractedHuijgen, R., Boekholdt, S. M., Arsenault, B. J., Bao, W., Davaine, J., Tabet, F., Petrides, F., Rye, K.-A., DeMicco, D. A., Barter, P. J., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.04.002 · Cited by 1
RetractedIPython: A System for Interactive Scientific Computing
Perez, F., & Granger, B. E. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.53 · Cited by 4,009 · cited 1×
“Common approaches to this include this use of knitr [63] and IPython Notebooks [64].”
MethodCockburn, A. (2001) · UCLA - Biblioteca de Ciencias y Tecnologia Felix Morales Bueno · Cited by 1,858
Verification and Validation in Scientific Computing
Oberkampf, W. L., & Roy, C. J. (2010) · doi:10.1017/cbo9780511760396 · Cited by 1,556 · cited 1×
“Exactly how they check correctness depends on the researcher’s understanding of the problem at hand [24, 32, 46].”
BackgroundDesign and code inspections to reduce errors in program development
Fagan, M. E. (1976) · IBM Systems Journal · doi:10.1147/sj.153.0182 · Cited by 1,492
Agile Software Development, Principles, Patterns, and Practices
Martin, R. (2002) · Cited by 1,466
Research works printed but not linked: 4
- Z Merali, 2010, Error: why scientific programming does not compute, Nature, 467, 775
- Baddeley A, Eysenck MW, Anderson MC (2009) Memory. New York: Psychology Press.
- Ray DS, Ray EJ (2009) Unix and Linux: visual quickstart guide. 4th edition. San Francisco: Peachpit Press.
- Kelly D, Sanders R (2008) Assessing the quality of scientific software. In: Proceedings First International Workshop on Software Engineering for Computational Science and Engineering. Available: http://secse08.cs.ua.edu/Papers/Kelly.pdf.
Cited by
Items 301–400 of 727
Ulian, G., & Valdrè, G. (2022) · Journal of Applied Crystallography · doi:10.1107/s1600576722000085 · Cited by 12
EESSI: A cross‐platform ready‐to‐use optimised scientific software stack
Dröge, B., Rusu, V. H., Hoste, K., van Leeuwen, C. M., O’Cais, A., & Röblitz, T. (2022) · Software Practice and Experience · doi:10.1002/spe.3075 · Cited by 16
“…since they frequently lack the necessary expertise to efficiently manage the wealth of tools, libraries and applications that they require, 1,8 and sometimes find themselves limited to the software that they can get working in the short term, or waiting until others can facilitate the…”
Backgroundvan Houdt, P. J., Ragunathan, S., Berks, M., Ahmed, Z., Kershaw, L. E., Gurney‐Champion, O. J., Tadimalla, S., Arvidsson, J., Yu, S. Y., Kallehauge, J. F., et al. (2023) · Magnetic Resonance in Medicine · doi:10.1002/mrm.29826 · Cited by 16
“20,21 Furthermore, sharing code in public repositories is not yet standard practice, but is important to improve the reproducibility of results.”
BackgroundDeardorff, A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0230697 · Cited by 13
Ten simple rules to increase computational skills among biologists with Code Clubs
Hagan, A. K., Lesniak, N. A., Balunas, M. J., Bishop, L., Close, W. L., Doherty, M. K., Elmore, A. G., Flynn, K. J., Hannigan, G. D., Koumpouras, C. C., et al. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008119 · Cited by 12
The SoftWipe tool and benchmark for assessing coding standards adherence of scientific software
Zapletal, A., Höhler, D., Sinz, C., & Stamatakis, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-89495-8 · Cited by 12
A Padawan Programmer’s Guide to Developing Software Libraries
Yurkovich, J. T., Yurkovich, B. J., Dräger, A., Palsson, B. Ø., & King, Z. A. (2017) · Cell Systems · doi:10.1016/j.cels.2017.08.003 · Cited by 15
Toffee – a highly efficient, lossless file format for DIA-MS
Tully, B. (2020) · Scientific Reports · doi:10.1038/s41598-020-65015-y · Cited by 11
“This approach is fundamental to writing robust software [11].”
BackgroundCurrent status in spatiotemporal analysis of contrast‐based perfusion MRI
Shalom, E. S., Khan, A. I., Loo, S. V., & Sourbron, S. P. (2023) · Magnetic Resonance in Medicine · doi:10.1002/mrm.29906 · Cited by 13
“66,67 Application of these and other software development guidelines 68-70 to new contributions within the field will help to accelerate the pace of progression.”
MethodBest publishing practices to improve user confidence in scientific software
Poisot, T. (2015) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.6084/m9.figshare.1434688.v1 · Cited by 11
“The practice of science is becoming increasingly reliant on software—despite the lack of formal training (Hastings et al. 2014, Wilson et al. 2014), upwards of 30% of scientists need to develop their own.”
BackgroundPawlik, A., van Gelder, C. W. G., Nenadić, A., Palagi, P. M., Korpelainen, E., Lijnzaad, P., Marek, D., Sansone, S., Hancock, J. M., & Goble, C. (2017) · F1000Research · doi:10.12688/f1000research.11718.1 · Cited by 14
“This approach has proven to be very successful, allowing for maximum inclusivity and production of high quality training materials avoiding redundancy in contents (Wilson et al., 2014; Wilson et al., 2016).”
BackgroundTruth, Proof, and Reproducibility: There’s No Counter-Attack for the Codeless
Gray, C. T., & Marwick, B. (2019) · Communications in computer and information science · doi:10.1007/978-981-15-1960-4_8 · Cited by 11
“For those practicing statistical computing, a conversation is emerging about what constitutes best practice [40].”
BackgroundState of the practice for mesh generation and mesh processing software
Smith, S., Lazzarato, D. A., & Carette, J. (2016) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2016.06.008 · Cited by 11
PyMICE: APython library for analysis of IntelliCage data
Dzik, J. M., Puścian, A., Harda, Z., Radwańska, K., & Łęski, S. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0907-5 · Cited by 15
“…may be reproduced with the Pweave tool (Pastell, 2015) v. 0.24, (Pastell & Kowalski, 2016) an interpreter of Python programming language v. 2.7.3, PyMICE v. 1.1.0, (Dzik, Łęski & Puścian, 2016) NumPy v. 1.6.1, matplitlib v. 1.1.1rc, dateutil v. 1.5 and pytz v. 2012c. (Wilson et al., 2014).”
InfluentialChallenges for Verifying and Validating Scientific Software in Computational Materials Science
Vogel, T., Druskat, S., Scheidgen, M., Draxl, C., & Grunske, L. (2019) · IEEE/ACM International Workshop on Software Engineering for Science (SE4Science) · doi:10.1109/se4science.2019.00010 · Cited by 17
“packages a-posteriori and in an automated manner [10].”
InfluentialThe future of activity space and health research
Fuller, D., & Stanley, K. G. (2019) · Health & Place · doi:10.1016/j.healthplace.2019.05.009 · Cited by 11
HOW TO MAKE A PIE: REPRODUCIBLE RESEARCH FOR EMPIRICAL ECONOMICS AND ECONOMETRICS
Orozco, V., Bontemps, C., Maigné, É., Piguet, V., Hofstetter, A., Lacroix, A., Levert, F., & Rousselle, J.-M. (2020) · Journal of Economic Surveys · doi:10.1111/joes.12389 · Cited by 14
“The rule “Make names consistent, distinctive and meaningful ” from Wilson et al. (2014) identifies several points to address when naming objects, variables and functions in a program.”
BackgroundInfluentialFRETBursts: An Open Source Toolkit for Analysis of Freely-Diffusing Single-Molecule FRETPreprint
Ingargiola, A., Lerner, E., Chung, S., Weiss, S., & Michalet, X. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/039198 · Cited by 11
“To minimize the possibility of bugs being introduced inadvertently [36], we employ modern software engineering techniques such as unit testing and continuous integration [28, 37].”
MethodChemTools: Gain Chemical Insight from Quantum Chemistry Calculations
Pujal, L., Tehrani, A. M., & Heidar‐Zadeh, F. (2022) · Wiley eBooks · doi:10.1002/9783527829941.ch32 · Cited by 11
Geiger, R. S., Sholler, D., Culich, A., Martinez, C. C., de la Guardia, F. H., Lanusse, F., Ottoboni, K., Stuart, M. A., Vareth, M., Varoquaux, N., et al. (2018) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/a7b3m · Cited by 11
A how-to guide for code sharing in biology
Abdill, R. J., Talarico, E., & Grieneisen, L. (2024) · PLoS Biology · doi:10.1371/journal.pbio.3002815 · Cited by 16
“…or requires very specific system specifications [31]. review are frequent topics of discussion [9,21,24,34–39], as is “defensive programming” [22], that is, performing what may feel like excessive validation of the inputs and outputs of functions to make sure unexpected states (such as a…”
BackgroundGood modelling software practices
Lemmen, C., & Sommer, P. S. (2024) · Ecological Modelling · doi:10.1016/j.ecolmodel.2024.110890 · Cited by 12
“Educating about good practices became central in projects such as the Software Carpentry (Wilson 2016), highlighting the utility of live coding, pair programming and “open everything”.”
BackgroundREST: Embracing the rust programming language for modern electronic structure theory
Li, Z., Gao, T., Wang, S., Bi, S., Feng, R., Zhu, Z., Zhao, Y., Yan, W., Yu, L., Gao, Q., et al. (2025) · Chinese Journal of Chemical Physics · doi:10.1063/1674-0068/cjcp2510156 · Cited by 10
Advantages, Disadvantages and Misunderstandings About Document Driven Design for Scientific Software
Smith, S., Jegatheesan, T., & Kelly, D. F. (2016) · Software Engineering for High Performance Computing in Computational Science and Engineering · doi:10.1109/se-hpccse.2016.010 · Cited by 12
Early Developmental Activities and Computing Proficiency
Cutts, Q. I., Patitsas, E., Cole, E., Donaldson, P., Alshaigy, B., Gutica, M., Hellas, A., Larraza-Mendiluze, E., McCartney, R., & Riedesel, C. P. (2018) · doi:10.1145/3174781.3174789 · Cited by 15
“It is recognized from literature [18, 31, 39, 80] and our practice that nowadays programs are written by specialists in other domains who do not identify themselves as programmers and may not have ever been in a programming class.”
BackgroundComputational experiment design for operations model simulation
Lara, J. D., Lee, J. T., Callaway, D. S., & Hodge, B. S. (2020) · Electric Power Systems Research · doi:10.1016/j.epsr.2020.106680 · Cited by 10
Haim, M. (2021) · Medien & Kommunikationswissenschaft · doi:10.5771/1615-634x-2021-1-65 · Cited by 11
“Hierbei kann etwa auf bestehende Daten und Software aufgebaut werden, beispielsweise im Rahmen von Replikationsstudien, die Studierende zur detaillierten Auseinandersetzung mit zugänglichen Ressourcen zwingen.”
BackgroundInfluentialArabas, S., Jarecka, D., Jaruga, A., & Fijałkowski, M. (2014) · Scientific Programming · doi:10.1155/2014/870146 · Cited by 10
Popp, J., & Biskup, T. (2022) · Chemistry - Methods · doi:10.1002/cmtd.202100097 · Cited by 12
Guru, S. M., Hanigan, I., Nguyen, H. A., Burns, E., Stein, J., Blanchard, W., Lindenmayer, D. B., & Clancy, T. (2016) · Ecological Informatics · doi:10.1016/j.ecoinf.2016.08.003 · Cited by 10
“…these challenges, researchers have recommended best practices for performing computational science (Sandve et al., 2013; Stodden and Miguez, 2014) (Wilson et al., 2014) (Nosek et al., 2015) (Kenall et al., 2015), and although we endorse these recommendations, we believe that the importance of the…”
BackgroundCaddy, R. V., & Schneider, E. E. (2024) · The Astrophysical Journal · doi:10.3847/1538-4357/ad464a · Cited by 13
“CI tools have expanded rapidly in functionality and popularity over the last 20 yr and their usefulness in scienti fi c software is well established ( Beck 1999; Wilson et al. 2014, 2017 ) .”
When Scientific Software Meets Software Engineering
Leroy, D., Sallou, J., Bourcier, J., & Combemale, B. (2021) · Computer · doi:10.1109/mc.2021.3102299 · Cited by 13
Documenting research software in engineering science
Hermann, S., & Fehr, J. (2022) · Scientific Reports · doi:10.1038/s41598-022-10376-9 · Cited by 18
“x Wilson G, Aruliah DA, et al.(24) x x Wilson G, Bryan J, et al.”
BackgroundInfluentialSome thoughts on best publishing practices for scientific software
White, E. P. (2015) · Ideas in Ecology and Evolution · doi:10.4033/iee.2015.8.9.c · Cited by 9
“These are all important recommendations that will help encourage the use of good practice in the development of scientific software (Jackson et al. 2011, Osborne et al. 2014, Wilson et al. 2014).”
BackgroundSelf-citationmacroeco: reproducible ecological pattern analysis in Python
Kitzes, J. A., & Wilber, M. (2015) · Ecography · doi:10.1111/ecog.01905 · Cited by 9
Development of a Mesoscale Inversion System for Estimating Continental‐Scale CO2 Fluxes
Wesloh, D., Lauvaux, T., & Davis, K. J. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001818 · Cited by 9
“Wilson et al. (2014) note the importance of tests to ensure the reliability of software (Heroux & Willenbring, 2009; Heaton & Carver, 2015; Wilson, 2006); the tests for this package can be run from the root directory using tox or “python setup.py test”.”
MethodGood Enough Practices in Scientific Computing
Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., & Teal, T. K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005510 · Cited by 8
“Two years ago a group of researchers involved in Software Carpentry(1) and Data Carpentry(2) wrote a paper called “Best Practices for Scientific Computing” [1].”
BackgroundSelf-citationTaxonomy of purposes, methods, and recommendations for vulnerability analysis
Bonham, N., Kasprzyk, J. R., & Zagona, E. A. (2024) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2024.106269 · Cited by 8
A practical guide to coding line-by-line trace gas absorption in Earth's atmosphere
Korkin, S. V., Sayer, A. M., Ibrahim, A. I., & Lyapustin, A. (2025) · Journal of Quantitative Spectroscopy and Radiative Transfer · doi:10.1016/j.jqsrt.2025.109345 · Cited by 8
Fripp, M. (2018) · Energy Sustainability and Society · doi:10.1186/s13705-018-0184-x · Cited by 8
Guidelines for collaborative development of sustainable data treatment software
Wuttke, J., Cottrell, S. P., Gonzalez, M. A., Kaestner, A. P., Markvardsen, A. J., Rod, T. H., Rozyczko, P. B., & Vardanyan, G. (2022) · Journal of Neutron Research · doi:10.3233/jnr-220002 · Cited by 11
“Furthermore, assertions are valuable as self-testing documentation [84,320]: Assert(x>0) is much better than a comment that says x is expected and required to be a nonzero positive number.”
BackgroundHuman Centered Data Science: Ungrading in an Introductory Data Science Course
Theobold, A. S. (2023) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3587102.3588816 · Cited by 8
State of the Practice for Lattice Boltzmann Method Software
Smith, S., Michalski, P., Carette, J., & Keshavarz‐Motamed, Z. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09981-2 · Cited by 8
“Although the advice to modular-ize research software to handle complexity is common [90, 137, 138], specific guidelines on how to divide the software into modules is less prevalent.”
Duo, H., Li, Y., Yang, L., Tao, J., Yang, Q., Xiao, Y., Sun, J., Li, L., Nie, X., Zhang, X., et al. (2024) · Genome biology · doi:10.1186/s13059-024-03290-y · Cited by 10
Campbell, E. C., Christensen, K. M., Nuwer, M., Ahuja, A., Boram, O., Liu, J., Miller, R., Osuna, I., & Riser, S. C. (2024) · Journal of Geoscience Education · doi:10.1080/10899995.2024.2384338 · Cited by 10
“…best practices, such as modularizing code, adhering to variable naming conventions, writing comments, and applying consistent style and formatting (Wilson et al., 2014), as well as effective visualization principles, including legibility and labeling (Hepworth et al., 2020) and considerations of…”
Simulation Design: Trans-Paradigm Best-Practice from Software Engineering
Rossiter, S. (2015) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2842 · Cited by 8
“4.12 Reproducibility is significantly aided by storing model code in a version control system (VCS)—see Wilson et al. (2014).”
BackgroundMethodInfluentialFourth Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE4)
Katz, D. S., Niemeyer, K. E., Gesing, S., Hwang, L., Bangerth, W., Hettrick, S., Idaszak, R., Salac, J., Hong, N. C., Núñez-Corrales, S., et al. (2018) · Journal of Open Research Software · doi:10.5334/jors.184 · Cited by 13
“2014 [47] – 8 high level categories • Butterfly: a paradigm towards stable bio and neuro informatics tools development [48] • CIG Software Development Best Practices – https://geodynamics.”
BackgroundMake like a glass frog: In support of increased transparency in herpetology
Marshall, B. M. (2021) · Herpetological Journal · doi:10.33256/31.1.3545 · Cited by 10
Python Framework for Wind Turbines Enabling Test Automation of MoWiT
Fricke, J., Wiens, M., Requate, N., & Leimeister, M. (2021) · Linköping electronic conference proceedings · doi:10.3384/ecp21181403 · Cited by 8
“The reduction of coupling in the code, aim for high flexibility, and coding principles like DRY (Wilson et al.2014) are helping to achieve that goal.”
BackgroundPolyglot Programming in Applications Used for Genetic Data Analysis
Nowak, R. M. (2014) · BioMed Research International · doi:10.1155/2014/253013 · Cited by 8
“They should use software engineering practices and tools to avoid common mistakes and to speed up the development tasks [4].”
MethodShould We Have Blind Faith in Bioinformatics Software? Illustrations from the SNAP Web-Based Tool
Robiou-du-Pont, S., Li, A., Christie, S., Sohani, Z. N., & Meyre, D. (2015) · PLoS ONE · doi:10.1371/journal.pone.0118925 · Cited by 8
Kuksenok, K., Aragon, C., Fogarty, J. A., Lee, C. P., & Neff, G. (2017) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-017-9285-x · Cited by 9
“Researchers adopt and adapt aspects of professionalized software engineering terms of art, tools, and best practices (e.g., Mislan et al. 2016; Wilson et al. 2014; Sletholt et al. 2011; Hannay et al. 2009).”
Backgroundbíogo: a simple high-performance bioinformatics toolkit for the Go languagePreprint
Kortschak, R. D., & Adelson, D. L. (2014) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/005033 · Cited by 8
“In addition to the issues raised above, a recent paper has highlighted the importance of good software engineering practices in scientific computing and the potential problems that may arise when these are not adopted (Wilson et al., 2014).”
BackgroundMethodInfluentialMake like a glass frog: In support of increased transparency in herpetologyPreprint
Marshall, B. M., & Strine, C. T. (2020) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/74frd · Cited by 8
Requirements Engineering for Research Software: A Vision
Bajraktari, A., Binder, M., & Vogelsang, A. (2024) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re59067.2024.00050 · Cited by 7
Automated Data Processing Workflows for Non‐Expert Users of NMR Facilities
Afrough, A., Pérez‐Mendigorri, M., & Vosegaard, T. (2025) · Magnetic Resonance in Chemistry · doi:10.1002/mrc.5540 · Cited by 7
Archiving Software Surrogates on the Web for Future Reference
Holzmann, H., Sperber, W., & Runnwerth, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-43997-6_17 · Cited by 7
“[2] published a study on scientific software.”
Biospytial: spatial graph-based computing for ecological Big Data
Molgora, J. M. E., Sedda, L., & Atkinson, P. M. (2020) · GigaScience · doi:10.1093/gigascience/giaa039 · Cited by 8
Formal Verification, Scientific Code, and the Epistemological Heterogeneity of Computational Science
Imbert, C., & Ardourel, V. (2022) · Philosophy of Science · doi:10.1017/psa.2022.78 · Cited by 7
“Remember that, in general, scientific code is already of low quality (Wilson et al. 2014, 1).”
BackgroundFARGOCPT: 2D Multiphysics code for simulating disk interactions with stars, planets, and particles
Rometsch, T., Jordan, L. M., Moldenhauer, T. W., Wehner, D., Restrepo, S. R., Müller, T. W. A., Picogna, G., Kley, W., & Dullemond, C. P. (2024) · Astronomy and Astrophysics · doi:10.1051/0004-6361/202348687 · Cited by 9
Building HPC Learning Pathways: Understanding our Community
Filinger, W., Mullen, J., Cohen, J., Wittke, S., & Backhaus, A. (2024) · Practice and Experience in Advanced Research Computing · doi:10.1145/3626203.3670513 · Cited by 7
“This HPC material has been augmented by training workshops from groups such as The Carpentries [5, 14] and CodeRefinery [3] that provide a starting point for developing computing skills.”
ECOTOXr: An R package for reproducible and transparent retrieval of data from EPA's ECOTOX database
de Vries, P. (2024) · Chemosphere · doi:10.1016/j.chemosphere.2024.143078 · Cited by 7
Assessing computational reproducibility in Behavior Research Methods
Ellis, D. A., Towse, J. N., Brown, O., Cork, A., Davidson, B. I., Devereux, S., Hinds, J. M., Ivory, M. R., Nightingale, S. J., Parry, D. A., et al. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02501-5 · Cited by 12
“…research has tended to focus on the success of open research practices that support data sharing (Bauer, 2022; Grahe, 2021; Towse et al., 2021a; Wilson et al., 2014), in software engineering other important aspects of hardware and software resilience are more prominent (e.g., Venters et al.,…”
BackgroundMolgenis-impute: imputation pipeline in a box
Kanterakis, A., Deelen, P., van Dijk, F., Byelas, H., Dijkstra, M., & Swertz, M. A. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1309-3 · Cited by 8
“The format for parameters and workflows is CSV. Simplicity and expandability was a primary development goal [43].”
MethodDataDeps.jl: Repeatable Data Setup for Reproducible Data Science
White, L., Togneri, R. B., Liu, W., & Bennamoun, M. (2019) · Journal of Open Research Software · doi:10.5334/jors.244 · Cited by 8
NESTOREv1.0: A MATLAB Package for Strong Forthcoming Earthquake Forecasting
Gentili, S., Brondi, P., & Giovambattista, R. D. (2023) · Seismological Research Letters · doi:10.1785/0220220327 · Cited by 7
de Buyl, P., Huang, M., & Deprez, L. (2017) · Journal of Open Research Software · doi:10.5334/jors.142 · Cited by 7
“The design choices are the results of the authors’ experience on scientific programming, with further motivation by the article on “Best Practices for Scientific Computing” by Wilson et al [11]: • The algorithms are implemented in Fortran modules in the src/ directory and shared between the simulations found in progra…”
MethodInfluentialSeismology software: state of the practice
Smith, S., Zeng, Z., & Carette, J. (2018) · Journal of Seismology · doi:10.1007/s10950-018-9731-3 · Cited by 7
Weisberg, S. M., Schinazi, V. R., Ferrario, A., & Newcombe, N. S. (2022) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0001146 · Cited by 9
Facilitating reproducible research by investigating computational metadata
Thavasimani, P., & Missier, P. (2016) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2016.7840958 · Cited by 7
Implementing Data Management Workflows in Research Groups Through Integrated Library Consultancy
Borycz, J. (2021) · Data Science Journal · doi:10.5334/dsj-2021-009 · Cited by 15
“Teaching and enforcing basic coding standards has clear benefits to the quality of research output (Li & Prasad, 2005; Popic, Velikic, Jaroslav, Spasic, & Vulic, 2018; Wilson et al., 2014).”
BackgroundRecognising and evaluating the effectiveness of extortion in the Iterated Prisoner’s Dilemma
Knight, V., Harper, M., Glynatsi, N. E., & Gillard, J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304641 · Cited by 8
“All of the code and data discussed in Results section is open sourced, archived, and written according to best scientific principles [16].”
BackgroundSchlögl, M., Waltersdorfer, L., Regner, P., Siposova, A., & Brenning, A. (2026) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2026.106962 · Cited by 6
Castet, É., Termoz-Masson, J., Vizcay, S., Delachambre, J., Myrodia, V., Aguilar, C., Matonti, F., & Kornprobst, P. (2024) · Journal of Vision · doi:10.1167/jov.24.4.19 · Cited by 9
“To work in an open-science framework, it has become essential to provide a version-control system (VCS) to keep track of the successive versions of an open-source code (Blischak, Davenport, & Wilson, 2016; Wilson et al., 2014).”
MethodAssessing and improving the quality of Fortran code in scientific software: FortranAnalyser
García‐Rodríguez, M., Añel, J. A., & Iglesias, J. R. (2024) · Software Impacts · doi:10.1016/j.simpa.2024.100692 · Cited by 6
Scalable data harmonization for single-cell image-based profiling with CytoTable
Bunten, D., Tomkinson, J., Serrano, E., Lippincott, M. J., Brewer, K. I., Rubinetti, V., Alquaddoomi, F., & Way, G. P. (2026) · Patterns · doi:10.1016/j.patter.2026.101514 · Cited by 8
“This conceptual model is grounded in long-term sustainable software practices 19,20 designed to promote maintainability alongside adaptability to avoid software collapse.”
Recommending Bug-fixing Comments from Issue Tracking Discussions in Support of Bug Repair
Krasniqi, R. (2021) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac51774.2021.00114 · Cited by 6
“Approach Project Short Description User Query Input Query Output Results Bug #, Comment Relevance Score [1-4]”
BackgroundMethodComputational Reproducibility: A Practical Framework for Data Curators
Sawchuk, S., & Khair, S. (2021) · Journal of eScience Librarianship · doi:10.7191/jeslib.2021.1206 · Cited by 8
poliastro: a Python library for interactive astrodynamics
Rodríguez, J., & Garrido, J. (2022) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-212e5952-015 · Cited by 6
Good Practices for High-Quality Scientific Computing
Dubey, A. (2022) · Computing in Science & Engineering · doi:10.1109/mcse.2023.3259259 · Cited by 6
“From time to time, quality-conscious individuals have published lists of good practices that would improve the quality of software and developer productivity.(4,5) This article provides onemore such list but with updated suggestions, motivated by the need to keep up with rapid changes in the computing industry.”
BackgroundNanocrystal programmable assembly beyond hard spheres (or shapes) and other (simple) potentials
Travesset, A. (2024) · Current Opinion in Solid State and Materials Science · doi:10.1016/j.cossms.2024.101159 · Cited by 8
A Scientist’s Perspective on Sustainable Scientific Software
Lenhardt, W. C. (2014) · Journal of Open Research Software · doi:10.5334/jors.ba · Cited by 6
“Best practices, for example as described by G. Wilson [5], include designing for people and not computers, defensive programming, optimize only after the software has been validated, use bugs as new test cases, and the use of software versioning systems (e.g., Subversion, GitHub).”
MethodA knowledge-based approach to scientific software development
Szymczak, D., Smith, S., & Carette, J. (2016) · doi:10.1145/2897676.2897680 · Cited by 6
“Ease of experimentation means that we also need facilities to parameterize algorithmic variabilities.”
Schiffthaler, B., Kostadima, M. A., Delhomme, N., & Rustici, G. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004937 · Cited by 6
“Depending on the target audience, introductions to Unix and R should be incorporated into a course’s program or training materials, possibly utilizing materials developed by the Software Carpentry Foundation [12,13].”
MethodoTree: Implementing experiments with dynamically determined data quantity
Konrad, M. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2018.10.006 · Cited by 6
Is epidemiology ready for Big Software?
Kerr, C. C. (2019) · Pathogens and Disease · doi:10.1093/femspd/ftz006 · Cited by 6
Python for Atmosphere and Ocean Scientists
Irving, D. (2019) · Journal of Open Source Education · doi:10.21105/jose.00037 · Cited by 6
“…would figure out how to apply them in their own context (i.e., in the context of netCDF files, the PyAOS stack, etc.), such a workshop would teach programming (G. Wilson et al., 2014), data management and reproducible research (Irving, 2016) skills in the atmosphere and ocean science context.”
BackgroundIrshad, O., & Khan, M. U. G. (2019) · Current Bioinformatics · doi:10.2174/1574893614666190409112025 · Cited by 6
A graduate student perspective on overcoming barriers to interacting with open-source software
Cereceda, O., & Quinn, D. (2020) · FACETS · doi:10.1139/facets-2019-0020 · Cited by 8
“Best practices of coding and OSS use are not discussed herein; these topics have been discussed in detail by Wilson et al. (2014) and Jiménez et al. (2017).”
MethodFluorescent Nuclei Measurements Macro (FNMM), a tool for automated cell quantification in ImageJ
Peterson, J. C., & DeRuiter, M. C. (2020) · Software Impacts · doi:10.1016/j.simpa.2020.100030 · Cited by 6
Ten quick tips for making things findable
Lin, S., Ali, I., & Wilson, G. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008469 · Cited by 9
“jpeg) will help everyone find what they need [22,23].”
BackgroundSelf-citationGetting Started with Scientific Python
Unpingco, J. (2016) · Springer eBooks · doi:10.1007/978-3-319-30717-6_1 · Cited by 6
Code Profiling in R: A Review of Existing Methods and an Introduction to Package GUIProfiler
Rubio, Á., & Villar-Rosety, F. M. (2015) · The R Journal · doi:10.32614/rj-2015-036 · Cited by 6
Ten quick tips for getting the most scientific value out of numerical data
Schwen, L. O., & Rueschenbaum, S. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006141 · Cited by 6
An open, scalable, and flexible framework for automated aerial measurement of field experiments
Schnaufer, C., Pistorius, J. L., & LeBauer, D. S. (2020) · Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping · doi:10.1117/12.2560008 · Cited by 6
“The foundation for our approach is outlined in “Best Practices for Scientific Computing.(14) Reproducibility is at the foundation of scientific discovery, and software tools are making it easier than ever for scientists to share and build on each other’s computational work.”
BackgroundAd hoc efforts for advancing data science education
DeMasi, O., Paxton, A., & Koy, K. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007695 · Cited by 7
“These ad hoc education efforts can take a variety of formats—including hours-long workshops, week-long boot camps, and semesterlong research projects—and are intended to complement existing formal educational structures [13–15] by embracing new tools and pedagogy as they emerge [16, 17].”
BackgroundDigital Engineering Development in an Academic CubeSat Project
Honoré‐Livermore, E., Birkeland, R., Bakken, S., Garrett, J. L., & Haskins, C. (2021) · Journal of Aerospace Information Systems · doi:10.2514/1.i010981 · Cited by 8
Marshall, D. P., Farah, E. S., Musselman, E. D., Pelot, N. A., & Grill, W. M. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013764 · Cited by 11
“Using developed libraries like PyFibers streamlines the model development process and minimizes potential coding errors from duplication of efforts [37,38].”
Rewiring climate modeling with machine learning emulators
Katwyk, P. V., Fox‐Kemper, B., Hewitt, H. T., & Bergen, K. J. (2026) · Communications Earth & Environment · doi:10.1038/s43247-026-03238-z · Cited by 10
Zhang, H., Maillo, A., Khan, S. A., Martinez‐de‐Morentin, X., Lehmann, R., Gómez-Cabrero, D., & Tegner, J. N. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.10.034 · Cited by 7
Metadata
- Authors
- Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., Waugh, B., White, E. P., & Wilson, P.
- Affiliations
- Mozilla Foundation (US)
- Ontario Tech University (CA)
- Michigan State University (US)
- Space Telescope Science Institute (US)
- University of Toronto (CA)
- Monterey Bay Aquarium Research Institute (US)
- University of California, Berkeley (US)
- University of British Columbia (CA)
- Queen Mary University of London (GB)
- University College London (GB)
- Utah State University (US)
- University of Wisconsin–Madison (US)
- Corresponding author
- Wilson, Greg
- ORCID
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-01-07
- Volume
- 12
- Issue
- 1
- Pages
- e1001745
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC3886731
- OpenAlex
- W2131966673
- Cited by
- 723
- References
- 75
- Citations by year
- 2026: 63 · 2025: 28 · 2024: 49 · 2023: 42 · 2022: 56 · 2021: 51 · 2020: 70 · 2019: 69 · 2018: 67 · 2017: 62 · 2016: 63 · 2015: 54 · 2014: 27
- FWCI
- 128.23
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion, cc-by)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (0953694, CAREER, EP/H043160/1)
- Alfred P. Sloan Foundation
- Natural Sciences and Engineering Research Council of Canada
- Engineering and Physical Sciences Research Council (EP/H043160/1, EP/H043101/1, EP/G007144/1)
- National Human Genome Research Institute (R01HG007513)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
- Keywords
- scientific computing, best practices
- MeSH terms
- Computing Methodologies, Humans, Software, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.