Computational Reproducibility via Containers in Psychology
Open accessAbstract
Scientific progress relies on the replication and reuse of research. Recent studies suggest, however, that sharing code and data does not suffice for computational reproducibility —defined as the ability of researchers to reproduce “par- ticular analysis outcomes from the same data set using the same code and software” (Fidler and Wilcox, 2018). To date, creating long-term computationally reproducible code has been technically challenging and time-consuming. This tutorial introduces Code Ocean, a cloud-based computational reproducibility platform that attempts to solve these problems. It does this by adapting software engineering tools, such as Docker, for easier use by scientists and scientific audiences. In this article, we first outline arguments for the importance of computational reproducibility, as well as some reasons why this is a nontrivial problem for researchers. We then provide a step-by-step guide to getting started with containers in research using Code Ocean. (Disclaimer: the authors all worked for Code Ocean at the time of this article’s writing.)
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CodeOcean capsules are known to be reproducible with little effort.
“To address this, we based our benchmark on CodeOcean capsules (See Figure 2), which are known to be reproducible with little effort (Clyburne-Sherin et al., 2019).”
From CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark (2024) · cites this paper as Clyburne-Sherin et al., 2019
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 2× · see citing works
“dent replication —has been a high-profile issue in light of work by the Center for Open Science (Collaboration, 2015; Nosek and Lakens, 2014).”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Docker: lightweight Linux containers for consistent development and deployment
Merkel, D. (2014) · Linux journal · Cited by 3,277 · see citing works
Jupyter Notebooks – a publishing format for reproducible computational workflows
Kluyver, T. A., Ragan-Kelley, B., Fernando, P., Granger, B., Bussonnier, M., Frederic, J., Kelley, K., Hamrick, J. B., Grout, J., Corlay, S., et al. (2016) · IOS Press eBooks · doi:10.3233/978-1-61499-649-1-87 · Cited by 3,191 · see citing works
Reproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · see citing works
An introduction to Docker for reproducible research
Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059 · see citing works
The Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963 · see citing works
Ten Simple Rules for Reproducible Computational Research
Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880 · see citing works
Registered Reports: A Method to Increase the Credibility of Published Results
Nosek, B. A., & Lakens, D. (2014) · Social Psychology · doi:10.1027/1864-9335/a000192 · Cited by 716 · see citing works
The contact hypothesis re-evaluated
Paluck, E., Green, D. P., & Green, S. (2018) · Behavioural Public Policy · doi:10.1017/bpp.2018.25 · Cited by 711 · see citing works
Self-citationDonoho, D. (2017) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2017.1384734 · Cited by 695 · see citing works
The poor availability of psychological research data for reanalysis.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.726 · Cited by 543 · see citing works
An empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369 · see citing works
Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., et al. (2018) · Royal Society Open Science · doi:10.1098/rsos.180448 · Cited by 336 · see citing works
Improving the Dependability of Research in Personality and Social Psychology
Funder, D. C., Levine, J. M., Mackie, D. M., Morf, C. C., Sansone, C., Vazire, S., & West, S. G. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313507536 · Cited by 254 · see citing works
Reproducibility of Scientific Results
Fidler, F. M., & Wilcox, J. R. (2018) · Cited by 199 · see citing works
Bumblebees learn foraging routes through exploitation-exploration cycles
Johnson, C., Prior, L., Archibald, S., Poulos, H., Barton, A., Williamson, G., & Bowman, D. (2018) · UTAS Research Repository · doi:10.6084/m9 · Cited by 190 · see citing works
Data Management Challenges of Data-Intensive Scientific Workflows
Deelman, E., & Chervenak, A. (2008) · IEEE International Symposium on Cluster Computing and the Grid (CCGrid) · doi:10.1109/ccgrid.2008.24 · Cited by 176 · see citing works
Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm
Vanpaemel, W., Vermorgen, M., Deriemaecker, L., & Storms, G. (2015) · Collabra · doi:10.1525/collabra.13 · Cited by 152 · see citing works
Terminologies for Reproducible ResearchPreprint
Barba, L. A. (2018) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2021.ii.2 · Cited by 136 · see citing works
A reanalysis of mouse ENCODE comparative gene expression data
Gilad, Y., & Mizrahi-Man, O. (2015) · F1000Research · doi:10.12688/f1000research.6536.1 · Cited by 100 · see citing works
An Introduction to Rocker: Docker Containers for R
Boettiger, C., & Eddelbuettel, D. (2017) · The R Journal · doi:10.32614/rj-2017-065 · Cited by 67 · see citing works
Sharing Data and Materials in Psychological Science
Lindsay, D. S. (2017) · Psychological Science · doi:10.1177/0956797617704015 · Cited by 62 · see citing works
Hung, L., Kristiyanto, D., Lee, S. B., & Yeung, K. Y. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152686 · Cited by 54 · see citing works
Why Most Of Psychology Is Statistically Unfalsifiable
Morey, R., & Lakens, D. (2016) · Zenodo · doi:10.5281/zenodo.838684 · Cited by 53 · see citing works
The hard road to reproducibility
Barba, L. A. (2016) · Science · doi:10.1126/science.354.6308.142 · Cited by 51 · see citing works
Liu, D. M., & Salganik, M. J. (2019) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023119849803 · Cited by 47 · see citing works
The Practice of Reproducible Research
(2019) · University of California Press eBooks · doi:10.1525/9780520967779 · Cited by 40 · see citing works
Lessons from a Decade of Replications at the <i>Quarterly Journal of Political Science</i>
Eubank, N. (2016) · Political Science and Politics · doi:10.1017/s1049096516000196 · Cited by 35 · see citing works
15 Years of Reproducible Research in Computational Harmonic Analysis
Donoho, D. L., Maleki, A., Rahman, I. U., Shahram, M., & Stodden, V. (2008) · Cited by 28 · see citing works
Silver, A. (2017) · Nature · doi:10.1038/546173a · Cited by 24 · see citing works
Push button replication: Is impact evaluation evidence for international development verifiable?
Wood, B. D. K., Müller, R., & Brown, A. N. (2018) · PLOS ONE · doi:10.1371/journal.pone.0209416 · Cited by 24 · see citing works
Reproducible Bioconductor workflows using browser-based interactive notebooks and containers
Almugbel, R., Hung, L., Hu, J., Almutairy, A., Ortogero, N., Tamta, Y., & Yeung, K. Y. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx120 · Cited by 24 · see citing works
The contact hypothesis re-evaluatedPreprint
Paluck, E. L., Green, S. A., & Green, D. (2017) · doi:10.31235/osf.io/w2jkf · Cited by 16 · see citing works
Self-citationMarwick, B., Rokem, A., & Станева, В. (2017) · doi:10.31235/osf.io/gne3w · Cited by 15 · see citing works
Liu, D. M., & Salganik, M. (2019) · Sage Journals Data · doi:10.25384/sage.c.4663025 · Cited by 11 · see citing works
Good Enough Practices in Scientific Computing
Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., & Teal, T. K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005510 · Cited by 8 · see citing works
Cooper, J. (2012) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2012.12.006 · Cited by 8 · see citing works
Enhancing pre-defined workflows with ad hoc analytics using Galaxy, Docker and JupyterPreprint
Grüning, B. A., Rasche, E. S., Rebolledo‐Jaramillo, B., Eberhard, C., Houwaart, T., Chilton, J. M., Coraor, N., Backofen, R., Taylor, J., & Nekrutenko, A. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/075457 · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 40 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Bayesian statistics and modelling
van de Schoot, R., Depaoli, S. A., King, R., Kramer, B. M. R., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J. E., et al. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-020-00001-2 · Cited by 1,185 · see citing works
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387 · see citing works
REFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J. R., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 192 · see citing works
A large-scale study on research code quality and execution
Trišović, A., Lau, M. K., Pasquier, T., & Crosas, M. (2022) · Scientific Data · doi:10.1038/s41597-022-01143-6 · Cited by 136 · see citing works
Bayesian statistics and modelling
van de Schoot, R., Depaoli, S. A., Gelman, A., King, R., Kramer, B. M. R., Märtens, K., Tadesse, M. G., Vannucci, M., Willemsen, J. E., & Yau, C. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-020-00003-0 · Cited by 118 · see citing works
Traceability for Trustworthy AI: A Review of Models and Tools
Mora‐Cantallops, M., Sánchez‐Alonso, S., García‐Barriocanal, E., & Sicilia, M. (2021) · Big Data and Cognitive Computing · doi:10.3390/bdcc5020020 · Cited by 92 · see citing works
Reproducibility in machine‐learning‐based research: Overview, barriers, and drivers
Semmelrock, H., Ross‐Hellauer, T., Kopeinik, S., Theiler, D., Haberl, A., Thalmann, S., & Kowald, D. (2025) · AI Magazine · doi:10.1002/aaai.70002 · Cited by 118 · see citing works
Why most psychological research findings are not even wrong
Scheel, A. M. (2022) · Infant and Child Development · doi:10.1002/icd.2295 · Cited by 91 · see citing works
Joint variational autoencoders for multimodal imputation and embedding
Kalafut, N. C., Huang, X. L., & Wang, D. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00663-z · Cited by 77 · see citing works
Practical Reproducibility in Geography and Geosciences
Nüst, D., & Pebesma, E. J. (2020) · Annals of the American Association of Geographers · doi:10.1080/24694452.2020.1806028 · Cited by 73 · see citing works
Containers for computational reproducibility
Moreau, D., Wiebels, K., & Boettiger, C. (2023) · Nature Reviews Methods Primers · doi:10.1038/s43586-023-00236-9 · Cited by 74 · see citing works
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81 · see citing works
Crüwell, S., Apthorp, D., Baker, B. J., Colling, L., Elson, M., Geiger, S. J., Lobentanzer, S., Monéger, J., Patterson, A., Schwarzkopf, D. S., et al. (2023) · Psychological Science · doi:10.1177/09567976221140828 · Cited by 54 · see citing works
Vega, J., Li, M., Aguillera, K., Goel, N., Joshi, E., Khandekar, K., Durica, K. C., Kunta, A. R., & Low, C. A. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.769823 · Cited by 54 · see citing works
Konkol, M., Nüst, D., & Goulier, L. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00095-y · Cited by 41 · see citing works
Mi, H., Sivagnanam, S., Ho, W. J., Zhang, S., Bergman, D. R., Deshpande, A. M., Baras, A. S., Jaffee, E. M., Coussens, L. M., Fertig, E. J., et al. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae421 · Cited by 38 · see citing works
In’nami, Y., Mizumoto, A., Plonsky, L., & Koizumi, R. (2022) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2022.100030 · Cited by 28 · see citing works
Bolibaugh, C., Vanek, N., & Marsden, E. J. (2021) · Bilingualism Language and Cognition · doi:10.1017/s1366728921000535 · Cited by 27 · see citing works
Advancing Computational Reproducibility in the Dataverse Data Repository Platform
Trišović, A., Durbin, P., Schlatter, T., Durand, G., Barbosa, S., Brooke, D., & Crosas, M. (2020) · International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS) · doi:10.1145/3391800.3398173 · Cited by 25 · see citing works
Model Reuse With Reduced Kernel Mean Embedding Specification
Wu, X.-Z., Xu, W., Liu, S., & Zhou, Z. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3086619 · Cited by 23 · see citing works
Space-based observations of tropospheric ethane map emissions from fossil fuel extraction
Brewer, J. F., Millet, D. B., Wells, K. C., Payne, V. H., Kulawik, S. S., Vigouroux, C., Cady‐Pereira, K., Pernak, R., & Zhou, M. (2024) · Nature Communications · doi:10.1038/s41467-024-52247-z · Cited by 23 · see citing works
Leveraging Containers for Reproducible Psychological Research
Wiebels, K., & Moreau, D. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211017853 · Cited by 20 · see citing works
Martín-Santamaría, R., Cavero, S., Herrán, A., Duarte, A., & Colmenar, J. M. (2022) · Evolutionary Computation · doi:10.1162/evco_a_00317 · Cited by 19 · see citing works
Nine best practices for research software registries and repositories
Garijo, D., Ménager, H., Hwang, L., Trišović, A., Hucka, M., Morrell, T. E., Allen, A., & Task Force on Best Practices for Software Registries (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1023 · Cited by 20 · see citing works
Container Orchestration on HPC Clusters
Piras, M. E., Pireddu, L., Moro, M., & Zanetti, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-34356-9_3 · Cited by 15 · see citing works
Bakdash, J. Z., Marusich, L. R., Kenworthy, J. B., Twedt, E., & Zaroukian, E. G. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.609647 · Cited by 14 · see citing works
A Universal Deep Room Acoustics Estimator
López, P. S., Callens, P., & Cerňak, M. (2021) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa52581.2021.9632738 · Cited by 12 · see citing works
Crüwell, S., Apthorp, D., Baker, B. J., Colling, L. J., Elson, M., Geiger, S. J., Lobentanzer, S., Monéger, J., Patterson, A., Schwarzkopf, D. S., et al. (2022) · doi:10.31234/osf.io/729qt · Cited by 11 · see citing works
Toward Reusable Science with Readable Code and Reproducibility
Bahaidarah, L., Hung, E., Oliveira, A. F. D. M., Penumaka, J., Rosario, L., & Trišović, A. (2022) · IEEE International Conference on eScience · doi:10.1109/escience55777.2022.00079 · Cited by 10 · see citing works
Operationalizing validity of empirical software engineering studies
Härtel, J., & Lämmel, R. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10370-3 · Cited by 10 · see citing works
Green, S. A., Smith, B., & Mathur, M. B. (2025) · Appetite · doi:10.1016/j.appet.2025.108233 · Cited by 10 · see citing works
Self-citationCharacterising reproducibility debt in scientific software: A systematic literature review
Hassan, Z., Treude, C., Norrish, M., Williams, G., & Potanin, A. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112327 · Cited by 12 · see citing works
A Personalized Learning Framework for Software Vulnerability Detection and Education
Taeb, M., & Chi, H. (2021) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic54682.2021.00032 · Cited by 9 · see citing works
Kovács, M., Varga, M. A., Dianovics, D., Poldrack, R. A., & Aczél, B. (2024) · Royal Society Open Science · doi:10.1098/rsos.240844 · Cited by 6 · see citing works
So, E., Yu, F., Wang, B., & Haibe‐Kains, B. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00688-4 · Cited by 7 · see citing works
Multilevel summation for periodic electrostatics using B-splines
Kaya, H., Hardy, D. J., & Skeel, R. D. (2021) · The Journal of Chemical Physics · doi:10.1063/5.0040925 · Cited by 7 · see citing works
Open and sustainable AI: challenges, opportunities and the road ahead in the life sciences
Farrell, G., Adamidi, E. S., Buono, R. A., Anton, M., Attafi, O. A., Capella-Gutiérrez, S., Capriotti, E., Castro, L. J., Cirillo, D., Crossman, L. C., et al. (2026) · Nature Methods · doi:10.1038/s41592-026-03037-6 · Cited by 10 · see citing works
Three Pillars of Practical Reproducibility
Keahey, K., Anderson, J., Powers, M., & Cooper, A. (2023) · IEEE International Conference on eScience · doi:10.1109/e-science58273.2023.10254846 · Cited by 6 · see citing works
Isnaldi, E., Richard, F., Schepper, M. D., Leduc, S., Maetens, M., Geukens, T., Baelen, K. V., Nguyen, H.-L., Rouas, G., Zoppoli, G. M., et al. (2022) · npj Breast Cancer · doi:10.1038/s41523-022-00453-7 · Cited by 6 · see citing works
Costa, L., Barbosa, S., & Cunha, J. (2025) · Future Generation Computer Systems · doi:10.1016/j.future.2025.107924 · Cited by 5 · see citing works
Containerization for creating reusable model code
Ferro, M. V., Lee, A., Pritchard, C., Barton, C. M., & Janssen, M. A. (2022) · Socio-Environmental Systems Modeling · doi:10.18174/sesmo.18074 · Cited by 5 · see citing works
Leveraging Containers for Reproducible Psychological ResearchPreprint
Wiebels, K., & Moreau, D. (2021) · doi:10.31234/osf.io/h7tkg · Cited by 5 · see citing works
Siegel, Z. S., Kapoor, S., Nadgir, N., Stroebl, B., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2409.11363 · Cited by 4 · see citing works
“To address this, we based our benchmark on CodeOcean capsules (See Figure 2), which are known to be reproducible with little effort (Clyburne-Sherin et al., 2019).”
Relating Sound and Sight in Simulated Environments
Tsang, K. Y.-T., & Mannion, D. J. (2022) · Multisensory Research · doi:10.1163/22134808-bja10082 · Cited by 4 · see citing works
Open Science in Three Acts: Foundations, Practice, and Implementation - Second Act
Limongi, R., & Rogers, P. (2025) · BAR - Brazilian Administration Review · doi:10.1590/1807-7692bar2025250116 · Cited by 4 · see citing works
Momeni, F., Khan, M. T., Kiesel, J., & Ross‐Hellauer, T. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746161 · Cited by 3 · see citing works
A Platform for the Reproducibility of Computational Experiments
Costa, L. (2022) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc53370.2022.9833132 · Cited by 3 · see citing works
Joint Variational Autoencoders for Multimodal Imputation and EmbeddingPreprint
Kalafut, N. C., Huang, X. L., & Wang, D. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.10.15.512388 · Cited by 3 · see citing works
Estimating lighting direction in scenes with multiple objects
Peterson, L. M., Kersten, D., & Mannion, D. J. (2023) · Attention Perception & Psychophysics · doi:10.3758/s13414-023-02718-0 · Cited by 2 · see citing works
Reducing the efforts to create reproducible analysis code with FieldTrip
van Es, M. W. J., Spaak, E., Schoffelen, J., & Oostenveld, R. (2024) · The Journal of Open Source Software · doi:10.21105/joss.05566 · Cited by 2 · see citing works
Wu, X., Luo, G., Dong, Z., Zheng, W., & Jia, G. (2024) · Genes · doi:10.3390/genes15040478 · Cited by 1 · see citing works
A Reinforcement Learning Based Approach to Solve Voltage Issues in Distribution Networks
Çakır, M. T., Nayır, H., Demіr, A., Kaya, H., & Ceylan, O. (2025) · IEEE International Conference on Compatibility, Power Electronics and Power Engineering · doi:10.1109/cpe-powereng63314.2025.11027244 · Cited by 1 · see citing works
Introduction to reproducible psychological researchPreprint
Kunisato, Y. (2019) · doi:10.31234/osf.io/x8js5 · Cited by 1 · see citing works
Ten simple rules for a successful international consortium in big data omics
Stobbe, M. D., González-Pérez, A., López-Bigas, N., & Gut, I. G. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010546 · Cited by 1 · see citing works
The Role of Third-Party Verification in Research ReproducibilityPreprint
Pérignon, C. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4496346 · Cited by 1 · see citing works
Gomes, D. A., Mestre, P., & Serôdio, C. (2023) · Preprints.org · doi:10.20944/preprints202310.0102.v1 · Cited by 1 · see citing works
Characterising Reproducibility Debt in Scientific Software:A Systematic Literature ReviewPreprint
Hassan, Z., Treude, C., Norrish, M., Williams, G., & Potanin, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4801433 · Cited by 1 · see citing works
Peer Review #2 of "Nine best practices for research software registries and repositories (v0.3)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.1023v0.3/reviews/2
Peer Review #1 of "Nine best practices for research software registries and repositories (v0.1)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.1023v0.1/reviews/1
Peer Review #2 of "Nine best practices for research software registries and repositories (v0.2)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.1023v0.2/reviews/2
Peer Review #2 of "Nine best practices for research software registries and repositories (v0.1)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.1023v0.1/reviews/2
Peer Review #1 of "Nine best practices for research software registries and repositories (v0.3)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.1023v0.3/reviews/1
Estimating lighting direction in scenes with multiple objectsPreprint
Peterson, L. M., Kersten, D., & Mannion, D. J. (2022) · doi:10.31234/osf.io/txy5u
Bhatt, S., Ferguson, N. M., Flaxman, S. R., Gandy, A., Mishra, S., & Scott, J. A. (2023) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1093/jrsssa/qnad055
So, E., Yu, F., Wang, B., & Haibe‐Kains, B. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.06.547938
Costa, L., Barbosa, S., & Cunha, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5032928
Visual features account for the appeal and visual interest of gerbera flowersPreprint
Patten, M. L. W., Mannion, D. J., Špehar, B., & Watson, T. (2024) · doi:10.31234/osf.io/jsrzq
Melchert, O., & Demircan, A. (2025) · Computer Physics Communications · doi:10.1016/j.cpc.2025.109851
Current limitations in functional connectivity assessment: suggestions for analysis improvements
Xu, T., Bayrak, R. G., Fischbach, A. K., Bridgeford, E., Vogelstein, J. T., Noble, S., & Anteraper, S. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-19099-5.00002-7
- Authors
- Clyburne-Sherin, A., Fei, X., & Green, S. A.
- Venue
- Meta-Psychology
- Publisher
- Linnaeus University
- Published
- 2019-11-12
- Volume
- 3
- ISSN
- 2003-2714 (ISSN-L)
- OpenAlex
- W2984916588
- Cited by
- 69
- References
- 40
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 10 · 2023: 13 · 2022: 19 · 2021: 12 · 2020: 4 · 2019: 2
- FWCI
- 10.96
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Meta-Psychology (publishedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- computational reproducibility, containers, Docker, research reproducibility, psychology research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
