Ten Simple Rules for the Open Development of Scientific Software
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Open-source software development has had significant impact, not only on society, but also on scientific research. Papers describing software published as open source are amongst the most widely cited publications (e.g., BLAST [1], [2] and Clustal-W [3]), suggesting many scientific studies may not have been possible without some kind of open software to collect observations, analyze data, or present results. It is surprising, therefore, that so few papers are accompanied by open software, given the benefits that this may bring. Publication of the source code you write not only can increase your impact [4], but also is essential if others are to be able to reproduce your results. Reproducibility is a tenet of computational science [5], and critical for pipelines employed in data-driven biological research. Publishing the source for the software you created as well as input data and results allows others to better understand your methodology, and why it produces, or fails to produce, expected results. Public release might not always be possible, perhaps due to intellectual property policies at your or your collaborators' institutes; and it is important to make sure you know the regulations that apply to you. Open licensing models can be incredibly flexible and do not always prevent commercial software release [5]. Simply releasing the source under an open license, however, is not sufficient if you wish your code to remain useful beyond its publication [6]. The sustainability of software after publication is probably the biggest problem faced by researchers who develop it, and it is here that participating in open development from the outset can make the biggest impact. Grant-based funding is often exhausted shortly after new software is released, and without support, in-house maintenance of the software and the systems it depends on becomes a struggle. As a consequence, the software will cease to work or become unavailable for download fairly quickly [7], which may contravene archival policies stipulated by your journal or funding body. A collaborative and open project allows you to spread the resource and maintenance load to minimize these risks, and significantly contributes to the sustainability of your software. If you have the choice, embracing an open approach to development has tremendous benefits. It allows you to build on the work of other scientists, and enables others to build on your own efforts. To make the development of open scientific software more rewarding and the experience of using software more positive, the following ten rules are intended to serve as a guide for any computational scientist.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Biopython: freely available python tools for computational molecular biology and bioinformatics
Cock, P. J. A., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., Friedberg, I., Hamelryck, T., Kauff, F., Wilczynski, B., et al. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp163 · Cited by 6,080
Best Practices for Scientific Computing
Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001745 · Cited by 723
Ten simple rules for a successful collaboration
Vicens, Q., & Bourne, P. E. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030044 · Cited by 92
URL decay in MEDLINE—a 4-year follow-up study
Wren, J. D. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn127 · Cited by 85
Ten Simple Rules for Getting Grants
Bourne, P. E., & Chalupa, L. M. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020012 · Cited by 48 · cited 1×
“Grant writing [24] is beyond the scope of the Ten Simple Rules presented here, but it is worth mentioning that if the rules laid out here are being followed, an open development community can ensure value beyond the lifetime of an award.”
Background
Cited by
MNE software for processing MEG and EEG data
Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C., Parkkonen, L., & Hämäläinen, M. S. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.10.027 · Cited by 1,951
KBase: The United States Department of Energy Systems Biology Knowledgebase
(2018) · doi:10.1038/nbt.4163 · Cited by 1,624
Weaver, S., Shank, S. D., Spielman, S. J., Li, M., Muse, S. V., & Kosakovsky Pond, S. L. (2018) · Molecular Biology and Evolution · doi:10.1093/molbev/msx335 · Cited by 983
Ten Simple Rules for Reproducible Computational Research
Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880
Ten quick tips for machine learning in computational biology
Chicco, D. (2017) · BioData Mining · doi:10.1186/s13040-017-0155-3 · Cited by 878
Metadata
- Authors
- Prlić, A., & Procter, J. B.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-12-06
- Volume
- 8
- Issue
- 12
- Pages
- e1002802
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2090469606
- Cited by
- 141
- References
- 5
- FWCI
- 23.36
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.