Reproducibility in Computational Neuroscience Models and Simulations
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Like all scientific research, computational neuroscience research must be reproducible. Big data science, including simulation research, cannot depend exclusively on journal articles as the method to provide the sharing and transparency required for reproducibility. METHODS: Ensuring model reproducibility requires the use of multiple standard software practices and tools, including version control, strong commenting and documentation, and code modularity. RESULTS: Building on these standard practices, model-sharing sites and tools have been developed that fit into several categories: 1) standardized neural simulators; 2) shared computational resources; 3) declarative model descriptors, ontologies, and standardized annotations; and 4) model-sharing repositories and sharing standards. CONCLUSION: A number of complementary innovations have been proposed to enhance sharing, transparency, and reproducibility. The individual user can be encouraged to make use of version control, commenting, documentation, and modularity in development of models. The community can help by requiring model sharing as a condition of publication and funding. SIGNIFICANCE: Model management will become increasingly important as multiscale models become larger, more detailed, and correspondingly more difficult to manage by any single investigator or single laboratory. Additional big data management complexity will come as the models become more useful in interpreting experiments, thus increasing the need to ensure clear alignment between modeling data, both parameters and results, and experiment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hucka, M., Finney, A., Sauro, H. M., Bolouri, H., Doyle, J. C., Kitano, H., Arkin, A. P., Bornstein, B. J., Bray, D., Cornish-Bowden, A., et al. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg015 · Cited by 3,060 · cited 2×
“VCell can import and export SBML reaction specifications [68] which then can be reproduced with other tools.”
BackgroundMethodNeuromorphic electronic systems
Mead, C. (1990) · Proceedings of the IEEE · doi:10.1109/5.58356 · Cited by 2,368 · cited 1×
“Still more problematic are simulations that use highly specialized systems such as neuromorphic chips [15], [16], or SpiNNaker chips [17], [18], hardware that is not commercially available.”
BackgroundReconstruction and Simulation of Neocortical Microcircuitry
Markram, H., Muller, E., Ramaswamy, S., Reimann, M. W., Abdellah, M., Sanchez, C. A., Ailamaki, A., Alonso-Nanclares, L., Antille, N., Arsever, S., et al. (2015) · Cell · doi:10.1016/j.cell.2015.09.029 · Cited by 1,557 · cited 1×
“molecular [22], [47], [48] to large networks [49], [50], with the ability to simulate across these multiple scales [51].”
BackgroundSynapse formation on neurons born in the adult hippocampus
Toni, N., Teng, E. M., Bushong, E. A., Aimone, J. B., Zhao, C., Consiglio, A., van Praag, H., Martone, M. E., Ellisman, M. H., & Gage, F. H. (2007) · Nature Neuroscience · doi:10.1038/nn1908 · Cited by 579 · cited 1×
“Experimentalists have probed the brain at many scales, from electron microscopy studies of synapses [1],”
BackgroundCellML: its future, present and past
Lloyd, C. M., Halstead, M. D. B., & Nielsen, P. F. (2004) · Progress in Biophysics and Molecular Biology · doi:10.1016/j.pbiomolbio.2004.01.004 · Cited by 499 · cited 5×
“CellML uses this feature to allow partial model descriptions for these qualitative models [102].”
BackgroundMethodInfluentialWeb-based kinetic modelling using JWS Online
Olivier, B. G., & Snoep, J. L. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth200 · Cited by 326 · cited 1×
“Some model repositories, such as JWS Online, address this challenge by running models on demand to regenerate data [140].”
BackgroundControlled vocabularies and semantics in systems biology
Courtot, M., Juty, N., Knüpfer, C., Waltemath, D., Zhukova, A., Dräger, A., Dumontier, M., Finney, A., Golebiewski, M., Hastings, J., et al. (2011) · Molecular Systems Biology · doi:10.1038/msb.2011.77 · Cited by 316 · cited 1×
“In the clinical realm, a consortium of major journals recently proposed requiring full data sharing for all clinical articles published [122].”
BackgroundParallel network simulations with NEURON
Migliore, M., Cannia, C., Lytton, W. W., Markram, H., & Hines, M. L. (2006) · Journal of Computational Neuroscience · doi:10.1007/s10827-006-7949-5 · Cited by 196
Self-citationAttention-like modulation of hippocampus place cell discharge
Fenton, A. A., Lytton, W. W., Barry, J. M., Lenck-Santini, P.-P., Zinyuk, L. E., Kubík, Š., Bureš, J., Poucet, B., Muller, R. U., & Olypher, A. V. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5576-09.2010 · Cited by 170 · cited 1×
“Human or animal behavioral research is the least replicable since it is never possible to control what the subject is thinking [11].”
BackgroundSelf-citationThe Physiome Model Repository 2
Yu, T., Lloyd, C. M., Nickerson, D. P., Cooling, M. T., Miller, A. K., Garny, A., Terkildsen, J. R., Lawson, J., Britten, R. D., Hunter, P. J., et al. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btq723 · Cited by 169
Cited by
NetPyNE, a tool for data-driven multiscale modeling of brain circuits
Dura-Bernal, S., Suter, B. A., Gleeson, P., Cantarelli, M., Quintana, A., Rodriguez, F., Kedziora, D. J., Chadderdon, G. L., Kerr, C. C., Neymotin, S. A., et al. (2019) · eLife · doi:10.7554/elife.44494 · Cited by 169
Self-citationImproving reproducibility in computational biology research
Papin, J. A., Mac Gabhann, F., Sauro, H. M., Nickerson, D., & Rampadarath, A. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007881 · Cited by 42
Recent advances in biomedical simulations: a manifesto for model engineering
Hellerstein, J. L., Gu, S., Choi, K., & Sauro, H. M. (2019) · F1000Research · doi:10.12688/f1000research.15997.1 · Cited by 20
Metadata
- Authors
- McDougal, R. A., Bulanova, A. S., & Lytton, W. W.
- Venue
- IEEE Transactions on Biomedical Engineering
- Published
- 2016-10
- Volume
- 63
- Issue
- 10
- Pages
- 2021-2035
- ISSN
- 0018-9294 (ISSN-L)
- 1558-2531
- OpenAlex
- W2320969362
- Cited by
- 70
- References
- 14
- FWCI
- 14.01
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reproducibility in Computational Neuroscience Models and Simulations. https://app.cochat.ai/papers/10.1109/tbme.2016.2539602
- The paper
- McDougal, R. A., Bulanova, A. S., & Lytton, W. W. (2016). Reproducibility in Computational Neuroscience Models and Simulations. IEEE Transactions on Biomedical Engineering. https://doi.org/10.1109/tbme.2016.2539602
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.