Best Practices for Scientific Computing
Open accessThe DOI matches its Crossref record; 73 of the 75 references it cites were found in a registry or an index.
Abstract
The Community Page is a forum for organizations and societies
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The creation of MIMIC-IV followed best practices in scientific computing23.”
From MIMIC-IV, a freely accessible electronic health record dataset (2023) · cites this paper as [23]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
5 retracted
Chang, G. A., Roth, C., Reyes, C. L., Pornillos, O., Chen, Y.-J., & Chen, A. P. (2006) · Science · doi:10.1126/science.314.5807.1875b · Cited by 137
RetractedRETRACTED: Structure of the multidrug resistance efflux transporter EmrE from Escherichia coli
Ma, C., & Chang, G. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0400137101 · Cited by 102
RetractedChang, G. A. (2007) · Journal of Molecular Biology · doi:10.1016/j.jmb.2003.05.001 · Cited by 22
RetractedMa, C., & Chang, G. A. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0700711104 · Cited by 18
RetractedHuijgen, R., Boekholdt, S. M., Arsenault, B. J., Bao, W., Davaine, J., Tabet, F., Petrides, F., Rye, K.-A., DeMicco, D. A., Barter, P. J., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.04.002 · Cited by 1
RetractedIPython: A System for Interactive Scientific Computing
Perez, F., & Granger, B. E. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.53 · Cited by 4,009 · cited 1×
“Common approaches to this include this use of knitr [63] and IPython Notebooks [64].”
MethodCockburn, A. (2001) · UCLA - Biblioteca de Ciencias y Tecnologia Felix Morales Bueno · Cited by 1,858
Verification and Validation in Scientific Computing
Oberkampf, W. L., & Roy, C. J. (2010) · doi:10.1017/cbo9780511760396 · Cited by 1,556 · cited 1×
“Exactly how they check correctness depends on the researcher’s understanding of the problem at hand [24, 32, 46].”
BackgroundDesign and code inspections to reduce errors in program development
Fagan, M. E. (1976) · IBM Systems Journal · doi:10.1147/sj.153.0182 · Cited by 1,492
Agile Software Development, Principles, Patterns, and Practices
Martin, R. (2002) · Cited by 1,466
Research works printed but not linked: 4
- Z Merali, 2010, Error: why scientific programming does not compute, Nature, 467, 775
- Baddeley A, Eysenck MW, Anderson MC (2009) Memory. New York: Psychology Press.
- Ray DS, Ray EJ (2009) Unix and Linux: visual quickstart guide. 4th edition. San Francisco: Peachpit Press.
- Kelly D, Sanders R (2008) Assessing the quality of scientific software. In: Proceedings First International Workshop on Software Engineering for Computational Science and Engineering. Available: http://secse08.cs.ua.edu/Papers/Kelly.pdf.
Cited by
Items 101–200 of 727
Berry, D. L., Blonquist, T. M., Patel, R. A., Halpenny, B., & McReynolds, J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4190 · Cited by 69
“Communication of research goals may be facilitated via improved use of shared artifacts such as models of study data and workflows [30].”
Anzt, H., Bach, F., Druskat, S., Löffler, F., Loewe, A., Renard, B. Y., Seemann, G., Struck, A., Achhammer, E., Aggarwal, P., et al. (2020) · F1000Research · doi:10.12688/f1000research.23224.2 · Cited by 98
Healthcare pathway discovery and probabilistic machine learning
Kempa-Liehr, A. W., Lin, C. Y.-C., Britten, R. D., Armstrong, D., Wallace, J., Mordaunt, D., & O’Sullivan, M. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104087 · Cited by 86
webchem : An R Package to Retrieve Chemical Information from the Web
Szöcs, E., Stirling, T., Scott, E. R., Scharmüller, A., & Schäfer, R. B. (2020) · Journal of Statistical Software · doi:10.18637/jss.v093.i13 · Cited by 70
“webchem follows best practices for scientific software (Wilson et al. 2014; Poisot 2015), namely: (i) a public available repository with easy collaboration and an issue tracker (via GitHub), (ii) a non-restrictive license, version control (git), (iii) an elaborate test-suite covering more than 90% of the relevant line…”
MethodMusselman, E. D., Cariello, J. E., Grill, W. M., & Pelot, N. A. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009285 · Cited by 78
Striving for transparent and credible research: practical guidelines for behavioral ecologists
Ihle, M., Winney, I. S., Krystalli, A., & Croucher, M. (2017) · Behavioral Ecology · doi:10.1093/beheco/arx003 · Cited by 75
“The essentials of these initially daunting technologies can be learned in just a few hours or days, as demonstrated by the international Software Carpentry Foundation (Wilson 2014).”
BackgroundManaging data locally to answer questions globally: The role of collaborative science in ecology
Aubin, I., Cardou, F., Boisvert‐Marsh, L., Garnier, É., Strukelj, M., & Munson, A. D. (2020) · Journal of Vegetation Science · doi:10.1111/jvs.12864 · Cited by 71
“Compounding the problem, literacy in new ecoinformatic tools remains a central challenge (Michener & Jones, 2012), and keeping up with best practices can seem like a moving target for most researchers (Wilson et al., 2014).”
BackgroundInvestment in the future electricity system - An agent-based modelling approach
Kraan, O., Kramer, G. J., & Nikolić, I. (2018) · Energy · doi:10.1016/j.energy.2018.03.092 · Cited by 67
“During the model development best practices for scientific computing have been pursued [51].”
BackgroundWong, T. E., Bakker, A. M. R., Ruckert, K. L., Applegate, P. J., Slangen, A. B. A., & Keller, K. M. (2017) · Geoscientific model development · doi:10.5194/gmd-10-2741-2017 · Cited by 74
Pauliuk, S., Majeau‐Bettez, G., Mutel, C., Steubing, B. R. P., & Stadler, K. (2015) · Journal of Industrial Ecology · doi:10.1111/jiec.12316 · Cited by 87
Bridging the Chasm: A Survey of Software Engineering Practice in Scientific Programming
Storer, T. (2017) · ACM Computing Surveys · doi:10.1145/3084225 · Cited by 93
Good scientific practice in EEG and MEG research: Progress and perspectives
Niso, G., Krol, L. R., Combrisson, E., Dubarry, A., Elliott, M., Clément, F., Héjja‐Brichard, Y., Herbst, S. K., Jerbi, K., Ković, V., et al. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119056 · Cited by 75
CancellationTools: All-in-one software for administration and analysis of cancellation tasks
Dalmaijer, E. S., der Stigchel, S. V., Nijboer, T. C. W., Cornelissen, T. H. W., & Husain, M. (2014) · Behavior Research Methods · doi:10.3758/s13428-014-0522-7 · Cited by 81
PySpike—A Python library for analyzing spike train synchrony
Mulansky, M., & Kreuz, T. (2016) · SoftwareX · doi:10.1016/j.softx.2016.07.006 · Cited by 66
“PySpike is written following modern coding standards and best practices in scientific computing [20], specifically adhering to the the official Style Guide for Python Code (PEP8)2.”
MethodOpen-Source Machine Learning in Computational Chemistry
Hagg, A., & Kirschner, K. N. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00643 · Cited by 71
Ten simple rules for documenting scientific software
Lee, B. D. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006561 · Cited by 60
“Write too many and readers will get lost in the sea of comments [4].”
BackgroundArabas, S., Jaruga, A., Pawłowska, H., & Grabowski, W. W. (2015) · Geoscientific model development · doi:10.5194/gmd-8-1677-2015 · Cited by 59
Sleep Spindles as an Electrographic Element: Description and Automatic Detection Methods
Coppieters ’t Wallant, D., Maquet, P., & Phillips, C. (2016) · Neural Plasticity · doi:10.1155/2016/6783812 · Cited by 59
“Moreover, different methods could directly be compared on the same databases with the same assessment method [134].”
Methodtidyhydat: Extract and Tidy Canadian Hydrometric Data
Albers, S. J. (2017) · The Journal of Open Source Software · doi:10.21105/joss.00511 · Cited by 59
“…analysis in a programming environment like R allows hydrologists the ability to create fully reproducible workflows, automate repetitive tasks and provide the same rigour to the data analysis process that hydrologists apply to field equipment and experimental design (Wilson et al. 2014).”
BackgroundOn best practices in the development of bioinformatics software
Leprevost, F. D. V., Barbosa, V. C., Francisco, E. L., Pérez‐Riverol, Y., & Carvalho, P. C. (2014) · Frontiers in Genetics · doi:10.3389/fgene.2014.00199 · Cited by 61
“Good programming practices range from documentation and code readability through design patterns and testing (Via et al., 2013; Wilson et al., 2014).”
BackgroundThe GenABEL Project for statistical genomics
Karssen, L. C., van Duijn, C. M., & Aulchenko, Y. S. (2016) · F1000Research · doi:10.12688/f1000research.8733.1 · Cited by 58
“These version control systems record any change to the files so they can easily be reviewed and reverted if necessary 7,29,30 .”
BackgroundIntegrating “Big Data” into Aquatic Ecology: Challenges and Opportunities
Durden, J. M., Luo, J. Y., Alexander, H., Flanagan, A. M., & Großmann, L. (2017) · Limnology and Oceanography Bulletin · doi:10.1002/lob.10213 · Cited by 63
Streamlining data-intensive biology with workflow systems
Reiter, T. E., Brooks, P., Irber, L. C., Joslin, S. E. K., Reid, C., Scott, C., Brown, C. T., & Pierce‐Ward, N. T. (2021) · GigaScience · doi:10.1093/gigascience/giaa140 · Cited by 65
“Building upon the rich literature of “best” and “good enough” practices for computational biology [8–10], we present a series of strategies and practices for adopting workflow systems to streamline data-intensive biology research.”
Self-citationFork It: Supporting Stateful Alternatives in Computational Notebooks
Weinman, N., Drucker, S. M., Barik, T., & DeLine, R. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445527 · Cited by 54
Open science, the replication crisis, and environmental public health
Hicks, D. (2021) · Accountability in Research · doi:10.1080/08989621.2021.1962713 · Cited by 63
“Specifically, Wilson et al. (2014) list 24 high-level principles of software engineering that, they argue, should be adopted by researchers using computationally intensive methods.”
BackgroundA model for streamlining and automating path exchange hybrid life cycle assessment
Stephan, A., Crawford, R. H., & Bontinck, P.-A. (2018) · The International Journal of Life Cycle Assessment · doi:10.1007/s11367-018-1521-1 · Cited by 65
A survey of biodiversity informatics: Concepts, practices, and challenges
Gadelha, L. M. R., Jr., de Siracusa, P. C., Dalcin, E., da Silva, L. A. E., Augusto, D. A., Krempser, E., Affe, H. M. D. J., Costa, R. L., Mondelli, M. L., Meirelles, P. M., et al. (2020) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1394 · Cited by 71
“Wilson et al. (2014) propose best practices for managing scientific computations.”
BackgroundLive cell microscopy: From image to insight
Cuny, A. P., Schlottmann, F. P., Ewald, J. C., Pelet, S. B., & Schmoller, K. M. (2022) · Biophysics Reviews · doi:10.1063/5.0082799 · Cited by 65
Maina, S., Donovan, N. J., Plett, K. L., Bogema, D. R., & Rodoni, B. (2024) · Frontiers in Horticulture · doi:10.3389/fhort.2024.1388028 · Cited by 68
Adapting bioinformatics curricula for big data
Greene, A. C., Giffin, K. A., Greene, C. S., & Moore, J. H. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbv018 · Cited by 55
A large-scale analysis of bioinformatics code on GitHub
Russell, P. H., Johnson, R. L., Ananthan, S., Harnke, B., & Carlson, N. E. (2018) · PLOS ONE · doi:10.1371/journal.pone.0205898 · Cited by 51
“The authors of [52] recommend specific software engineering practices for scientific computing.”
Wildlife biology, big data, and reproducible research
Lewis, K. P., Wal, E. V., & Fifield, D. A. (2018) · Wildlife Society Bulletin · doi:10.1002/wsb.847 · Cited by 59
van Zelst, I., Crameri, F., Pusok, A. E., Glerum, A., Dannberg, J., & Thieulot, C. (2022) · Solid Earth · doi:10.5194/se-13-583-2022 · Cited by 66
Principles for data analysis workflows
Stoudt, S., Vásquez, V., & Martinez, C. C. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008770 · Cited by 64
“Domain scientists seeking to wade into computational tool development are well advised to review the guidelines described in Gruning and colleagues [2] in addition to more traditional software development resources and texts such as Clean Code [44], Refactoring [45], and Best Practices in Scientific Computing [24].”
BackgroundMethodCiw: An open-source discrete event simulation library
Palmer, G., Knight, V., Harper, P., & Hawa, A. L. (2018) · Journal of Simulation · doi:10.1080/17477778.2018.1473909 · Cited by 56
Simple and extensible plate and shell finite element models through automatic code generation tools
Hale, J., Brunetti, M., Bordas, S., & Maurini, C. (2018) · Computers & Structures · doi:10.1016/j.compstruc.2018.08.001 · Cited by 71
“FEniCS-Shells follows the Best Practices for Scientific Computing [96] as closely as possible, including using version control, continuous integration and testing, repeatable computing environments [40] and providing thorough documentation.”
MethodComputational and Informatic Advances for Reproducible Data Analysis in Neuroimaging
Poldrack, R. A., Gorgolewski, K. J., & Varoquaux, G. P. (2019) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-072018-021237 · Cited by 61
Enforcing public data archiving policies in academic publishing: A study of ecology journals
Sholler, D., Ram, K., Boettiger, C., & Katz, D. S. (2019) · Big Data & Society · doi:10.1177/2053951719836258 · Cited by 59
Biological data sciences in genome researchPreprint
Schatz, M. C. (2015) · Genome Research · doi:10.1101/gr.191684.115 · Cited by 54
“…are not designed for this level of parallelism, needing new higher-level and easy-to-use workflow systems to orchestrate the scheduling and management of resources (Afgan et al. 2011), as well as improved software engineering practices to build more robust software (Wilson et al. 2014).”
BackgroundData science vs. statistics: two cultures?
Carmichael, I. D., & Marron, J. S. (2018) · Japanese Journal of Statistics and Data Science · doi:10.1007/s42081-018-0009-3 · Cited by 68
“Software engineers have built up best practices for programming (Wilson et al, 2014, 2017), which can be helpful for data analysts to learn, but may need to be adapted in some cases 40 .”
BackgroundDeveloping a modern data workflow for regularly updated data
Yenni, G. M., Christensen, E. M., Bledsoe, E. K., Supp, S. R., Diaz, R. M., White, E. P., & Ernest, S. K. M. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000125 · Cited by 34
Self-citationRöttger, M. C., Sanner, A., Thimons, L. A., Junge, T., Gujrati, A., Monti, J. M., Nöhring, W. G., Jacobs, T. D. B., & Pastewka, L. (2022) · Surface Topography Metrology and Properties · doi:10.1088/2051-672x/ac860a · Cited by 57
“We follow established practices in development of scientific software [86], such as continuous integration with an appropriate automated testing framework to ensure software quality.”
MethodKAnalyze: a fast versatile pipelined K-mer toolkit
Audano, P. A., & Vannberg, F. O. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu152 · Cited by 50
“Received on December 2, 2013; revised on January 24, 2014; accepted on March 12, 2014”
BackgroundLevick, B., South, A., & Hastings, I. M. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005327 · Cited by 52
CSDMS: a community platform for numerical modeling of Earth surface processes
Tucker, G. E., Hutton, E. W. H., Piper, M. D., Campforts, B., Gan, T., Barnhart, K. R., Kettner, A. J., Overeem, I., Peckham, S. D., McCready, L., et al. (2022) · Geoscientific model development · doi:10.5194/gmd-15-1413-2022 · Cited by 56
Packaging data analytical work reproducibly using R (and friends)Preprint
Marwick, B., Boettiger, C., & Mullen, L. A. (2017) · doi:10.7287/peerj.preprints.3192v1 · Cited by 50
Antunes, B. A., & Hill, D. R. C. (2024) · Computer Science Review · doi:10.1016/j.cosrev.2024.100655 · Cited by 78
Ten simple rules on writing clean and reliable open-source scientific software
Hunter-Zinck, H., de Siqueira, A. F., Vásquez, V., Barnes, R. J., & Martinez, C. C. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009481 · Cited by 63
“That being said, the conversation on software development for scientific research has shifted from “best” practices [28] to “good enough” practices [29].”
BackgroundThe State of Software for Evolutionary Biology
Darriba, D., Flouri, T., & Stamatakis, A. (2018) · Molecular Biology and Evolution · doi:10.1093/molbev/msy014 · Cited by 48
“Wilsonetal.”
Rampant software errors may undermine scientific results
Soergel, D. A. W. (2015) · F1000Research · doi:10.12688/f1000research.5930.2 · Cited by 57
Kraan, O., Kramer, G. J., Nikolić, I., Chappin, E. J. L., & Koning, V. (2019) · Energy Policy · doi:10.1016/j.enpol.2019.04.016 · Cited by 47
Fehr, J., 1 Institute of Engineering and Computational Mechanics at the University of Stuttgart, P. 9., Heiland, J., Himpe, C., & Saak, J. (2016) · AIMS Mathematics · doi:10.3934/math.2016.3.261 · Cited by 40
“And the best practices in [52] summarize code development fundamentals.”
Administrative social science data: The challenge of reproducible research
Playford, C., Gayle, V., Connelly, R., & Gray, A. J. G. (2016) · Big Data & Society · doi:10.1177/2053951716684143 · Cited by 52
“It is reported that these activities have proven to be very successful in some scientific areas (Goble, 2014; Wilson et al., 2014).”
BackgroundAdams, J. Y., Lieng, M. K., Kuhn, B. T., Rehm, G. B., Guo, E. C., Taylor, S. L., Delplanque, J., & Anderson, N. (2017) · Scientific Reports · doi:10.1038/s41598-017-15052-x · Cited by 47
Hagen, O. (2022) · Ecography · doi:10.1111/ecog.06132 · Cited by 50
What makes computational open source software libraries successful?
Bangerth, W., & Heister, T. (2013) · Computational Science & Discovery · doi:10.1088/1749-4699/6/1/015010 · Cited by 47
“It is curious to note that neither [37] nor books on numerical methods pay much attention to anything that goes beyond just writing and debugging the actual code. a project, its documentation, willingness to answer questions, etc.”
BackgroundDynamic Workflows for Routine Materials Discovery in Surface Science
Tran, K., Palizhati, A., Back, S., & Ulissi, Z. W. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00386 · Cited by 44
Naming the Pain in Developing Scientific Software
Wiese, I., Polato, I., & Pinto, G. (2019) · IEEE Software · doi:10.1109/ms.2019.2899838 · Cited by 51
Jug: Software for Parallel Reproducible Computation in Python
Coelho, L. P. (2017) · Journal of Open Research Software · doi:10.5334/jors.161 · Cited by 39
“Jug follows best practices in the field [60, 63] and includes a full test suite (>100 tests) with continuous integration using the Travis service.”
MethodEnhancing Climate Change Research With Open Science
Tai, T. C., & Robinson, J. P. W. (2018) · Frontiers in Environmental Science · doi:10.3389/fenvs.2018.00115 · Cited by 46
“Open science workflows facilitate large collaborations (e.g., GitHub, Open Science Framework; Ram, 2013; Wilson et al., 2014)”
BackgroundComputing Workflows for Biologists: A Roadmap
Shade, A., & Teal, T. K. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002303 · Cited by 40
“Recent works have promoted methods for reproducibility in computational research [2,3], provided guidelines for writing software and tools [1,4 – 6] and advocated for training for the next generation of scientists in computing [7 – 9].”
BackgroundMethodDrawing a line: Setting guidelines for digital image processing in scientific journal articles
Frow, E. (2012) · Social Studies of Science · doi:10.1177/0306312712444303 · Cited by 47
Tandon, N., & Tandon, R. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2019.08.032 · Cited by 41
Goodman, K. W. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781139600330 · Cited by 49
ss3sim: An R Package for Fisheries Stock Assessment Simulation with Stock Synthesis
Anderson, S. C., Monnahan, C. C., Johnson, K. F., Ono, K., & Valero, J. L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092725 · Cited by 45
“Second, the components of the simulation framework must work together as expected (integration tests [30]).”
BackgroundPowerSystems.jl — A power system data management package for large scale modeling
Lara, J. D., Barrows, C., Thom, D., Krishnamurthy, D., & Callaway, D. S. (2021) · SoftwareX · doi:10.1016/j.softx.2021.100747 · Cited by 53
A review of open software resources in python for electrical resistivity modelling
Doyoro, Y. G., Chang, P., Puntu, J. M., Lin, D.-J., Huu, T. V., Rahmalia, D. A., & Shie, M.-S. (2022) · Geoscience Letters · doi:10.1186/s40562-022-00214-1 · Cited by 41
“The development of SimPEG focused on modularity, usability, documentation, and extensive unit testing that can help as an interface create other inversion code in the geoscientific community (Cockett et al. 2015; Wilson et al. 2014).”
BackgroundPrinciples of resilient coding for plant ecophysiologists
Stinziano, J. R., Roback, C., Sargent, D., Murphy, B. K., Hudson, P. J., & Muir, C. D. (2021) · AoB Plants · doi:10.1093/aobpla/plab059 · Cited by 37
“As well, guides to best practices for scientific computing exist (see Wilson et al. 2014 for a list of best practices).”
Hsu, L., Martin, R. L., McElroy, B. J., Litwin-Miller, K., & Kim, W. (2015) · Geomorphology · doi:10.1016/j.geomorph.2015.03.039 · Cited by 37
Teaching R in the undergraduate ecology classroom: approaches, lessons learned, and recommendations
Auker, L. A., & Barthelmess, E. L. (2020) · Ecosphere · doi:10.1002/ecs2.3060 · Cited by 36
VISIR-2: ship weather routing in Python
Mannarini, G., Salinas, M. L., Carelli, L., Petacco, N., & Orović, J. (2024) · Geoscientific model development · doi:10.5194/gmd-17-4355-2024 · Cited by 42
Opportunities for agent‐based modelling in human dimensions of fisheries
Burgess, M. G., Carrella, E., Drexler, M., Axtell, R. L., Bailey, R. M., Watson, J. R., Cabral, R. B., Clemence, M., Costello, C., Dorsett, C., et al. (2020) · Fish and Fisheries · doi:10.1111/faf.12447 · Cited by 41
Practices in source code sharing in astrophysics
Shamir, L., Wallin, J. F., Allen, A., Berriman, G. B., Teuben, P. J., Nemiroff, R. J., Mink, J. D., Hanisch, R. J., & DuPrie, K. (2013) · Astronomy and Computing · doi:10.1016/j.ascom.2013.04.001 · Cited by 35
Dealing with software complexity in individual‐based models
Vedder, D., Ankenbrand, M. J., & Cabral, J. S. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13716 · Cited by 37
The NIH BD2K center for big data in translational genomics
Paten, B., Diekhans, M., Druker, B. J., Friend, S. H., Guinney, J., Gassner, N. C., Guttman, M., Kent, W. J., Mantey, P. E., Margolin, A. A., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv047 · Cited by 34
A Minimum Standard for Publishing Computational Results in the Weather and Climate Sciences
Irving, D. (2015) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-15-00010.1 · Cited by 34
“There are numerous authors who suggest that the best practices adopted by IS2015 (e.g., scripting, version control) save time in the long run (e.g., Sandve et al. 2013; Wilson et al. 2014), which means that once researchers have learned and adopted these practices they may actually save time.”
BackgroundHistorical Collaborative Geocoding
Cura, R., Duménieu, B., Abadie, N., Costes, B., Perret, J., & Gribaudi, M. (2018) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi7070262 · Cited by 49
Best practices for workflow design: how to prevent workflow decay
Hettne, K. M., Wolstencroft, K., Belhajjame, K., Goble, C., Mina, E., Dharuri, H. K., Verdes-Montenegro, L. L., Garrido, J., Roure, D. D., & Roos, M. (2012) · Research Explorer (The University of Manchester) · Cited by 33
PCMSolver: An open‐source library for solvation modeling
Remigio, R. D., Steindal, A. H., Mozgawa, K., Weijo, V., Cao, H., & Frediani, L. (2018) · International Journal of Quantum Chemistry · doi:10.1002/qua.25685 · Cited by 35
Provoking a Cultural Shift in Data Quality
McCord, S. E., Webb, N. P., Zee, J. W. V., Burnett, S. H., Christensen, E. M., Courtright, E. M., Laney, C., Lunch, C. K., Maxwell, C. M., Karl, J. W., et al. (2021) · BioScience · doi:10.1093/biosci/biab020 · Cited by 36
Machine learning in psychiatry- standards and guidelines
Tandon, N., & Tandon, R. (2019) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2019.09.009 · Cited by 34
Methods for training collaborative biostatisticians
Pomann, G.-M., Boulware, L. E., Cayetano, S. M., Desai, M., Enders, F. T., Gallis, J. A., Gelfond, J., Grambow, S. C., Hanlon, A. L., Hendrix, A., et al. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.518 · Cited by 32
Software citation, reuse and metadata considerations: An exploratory study examining LAMMPS
Li, K., Lin, X., & Greenberg, J. (2016) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2016.14505301072 · Cited by 32
“This concept is also covered by studies on software sustainability, most of which examine reuse from the developer’s perspective, i.e., how to take reusability into consideration when the software is being developed (e.g., Crouch et al., 2013; Stewart, Almes & Wheeler, 2010; Wilson et al., 2012).”
MethodCollating stock assessment packages to improve stock assessments
Dichmont, C. M., Deng, R. A., Dowling, N. A., & Punt, A. E. (2020) · Fisheries Research · doi:10.1016/j.fishres.2020.105844 · Cited by 34
Geiger, R. S., Varoquaux, N., Mazel-Cabasse, C., & Holdgraf, C. (2018) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-018-9333-1 · Cited by 32
“e open science and reproducibility movements, as the softwareisfreetouse,opentoreviewandmodification,andletsresearchersshare every step taken in processing and analyzing the data (Sandve et al. 2013; Wilson et al. 2014; Kitzes et al. 2018). The rise of the Python and R scripting languages in academic and industry data…”
BackgroundComputer Simulation in Low‐Temperature Plasma Physics: Future Challenges
Turner, M. M. (2016) · Plasma Processes and Polymers · doi:10.1002/ppap.201600121 · Cited by 32
Five Guiding Principles to Make Jupyter Notebooks Fit for Earth Observation Data Education
Wagemann, J., Fierli, F., Mantovani, S., Siemen, S., Seeger, B., & Bendix, J. (2022) · Remote Sensing · doi:10.3390/rs14143359 · Cited by 31
Herr, J. R., Öpik, M., & Hibbett, D. S. (2014) · New Phytologist · doi:10.1111/nph.13180 · Cited by 30
Visser, M. D., McMahon, S. M., Merow, C., Dixon, P. M., Record, S., & Jongejans, E. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004140 · Cited by 31
Bond‐Lamberty, B., Dorheim, K., Cui, R. Y., Horowitz, R., Snyder, A., Calvin, K., Feng, L., Hoesly, R., Horing, J., Kyle, P. G., et al. (2019) · Journal of Open Research Software · doi:10.5334/jors.232 · Cited by 31
3D electromagnetic modelling and inversion: a case for open source
Oldenburg, D. W., Heagy, L. J., Kang, S., & Cockett, R. (2019) · Exploration Geophysics · doi:10.1080/08123985.2019.1580118 · Cited by 30
“Peer review of code before it is incorporated into the main code base can reduceambiguities andunnecessary complexity, aswell as bugs in the implementation (Wilson et al. 2014).”
Backgroundlibmpdata++ 1.0: a library of parallel MPDATA solvers for systems of generalised transport equations
Jaruga, A., Arabas, S., Jarecka, D., Pawłowska, H., Smolarkiewicz, P. K., & Waruszewski, M. (2015) · Geoscientific model development · doi:10.5194/gmd-8-1005-2015 · Cited by 30
Synthetic Biology: fostering the cyber-biological revolution
Peccoud, J. (2016) · Synthetic Biology · doi:10.1093/synbio/ysw001 · Cited by 37
Oughton, E. J. (2022) · Telematics and Informatics · doi:10.1016/j.tele.2022.101908 · Cited by 39
ChemKED: A Human‐ and Machine‐Readable Data Standard for Chemical Kinetics Experiments
Weber, B. W., & Niemeyer, K. E. (2018) · International Journal of Chemical Kinetics · doi:10.1002/kin.21142 · Cited by 29
Machine Learning for Understanding Inland Water Quantity, Quality, and Ecology
Appling, A., Oliver, S. K., Read, J. S., Sadler, J. M., & Zwart, J. A. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-819166-8.00121-3 · Cited by 34
Direct and indirect effects of work‒family enrichment: role of gender role ideology
Bansal, N., & Agarwal, U. A. (2019) · International Journal of Productivity and Performance Management · doi:10.1108/ijppm-10-2018-0370 · Cited by 32
Taylor, I., Doering, K., Johnson, K. F., Wetzel, C. R., & Stewart, I. J. (2021) · Fisheries Research · doi:10.1016/j.fishres.2021.105924 · Cited by 40
MzJava: An open source library for mass spectrometry data processing
Horlacher, O., Nikitin, F., Alocci, D., Mariethoz, J., Müller, M., & Lisacek, F. (2015) · Journal of Proteomics · doi:10.1016/j.jprot.2015.06.013 · Cited by 30
“Development methodology The methodology that was employed to develop MzJava follows best practice for scientific computing [32][33][34][35] and is influenced by agile software development, especially test driven development (TDD) [36] and continuous integration (CI) [37].”
MethodBorries, C., Sandel, A. A., Koenig, A., Fernández‐Duque, E., Kamilar, J. M., Amoroso, C. R., Barton, R. A., Bray, J., Fiore, A. D., Gilby, I. C., et al. (2016) · Evolutionary Anthropology Issues News and Reviews · doi:10.1002/evan.21502 · Cited by 30
Metadata
- Authors
- Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., Waugh, B., White, E. P., & Wilson, P.
- Affiliations
- Mozilla Foundation (US)
- Ontario Tech University (CA)
- Michigan State University (US)
- Space Telescope Science Institute (US)
- University of Toronto (CA)
- Monterey Bay Aquarium Research Institute (US)
- University of California, Berkeley (US)
- University of British Columbia (CA)
- Queen Mary University of London (GB)
- University College London (GB)
- Utah State University (US)
- University of Wisconsin–Madison (US)
- Corresponding author
- Wilson, Greg
- ORCID
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-01-07
- Volume
- 12
- Issue
- 1
- Pages
- e1001745
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC3886731
- OpenAlex
- W2131966673
- Cited by
- 723
- References
- 75
- Citations by year
- 2026: 63 · 2025: 28 · 2024: 49 · 2023: 42 · 2022: 56 · 2021: 51 · 2020: 70 · 2019: 69 · 2018: 67 · 2017: 62 · 2016: 63 · 2015: 54 · 2014: 27
- FWCI
- 128.23
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion, cc-by)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (0953694, CAREER, EP/H043160/1)
- Alfred P. Sloan Foundation
- Natural Sciences and Engineering Research Council of Canada
- Engineering and Physical Sciences Research Council (EP/H043160/1, EP/H043101/1, EP/G007144/1)
- National Human Genome Research Institute (R01HG007513)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
- Keywords
- scientific computing, best practices
- MeSH terms
- Computing Methodologies, Humans, Software, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.