Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The widespread reluctance to share published research data is often hypothesized to be due to the authors' fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically. METHODS AND FINDINGS: We related the reluctance to share research data for reanalysis to 1148 statistically significant results reported in 49 papers published in two major psychology journals. We found the reluctance to share data to be associated with weaker evidence (against the null hypothesis of no effect) and a higher prevalence of apparent errors in the reporting of statistical results. The unwillingness to share data was particularly clear when reporting errors had a bearing on statistical significance. CONCLUSIONS: Our findings on the basis of psychological papers suggest that statistical results are particularly hard to verify when reanalysis is more likely to lead to contrasting conclusions. This highlights the importance of establishing mandatory data archiving policies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To avoid bias in the prevalence of reporting errors, we performed a validity study of statcheck, in which we compared statcheck’s results with the results of Wicherts, Bakker, and Molenaar (2011), who performed a manual search for and verification of reporting errors in a sample of 49 articles.”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Wicherts, Bakker, and Molenaar (2011)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, Wicherts et al. included all results reported in the text or a table in the Results section of an article.”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Wicherts et al., 2011
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Moreover, Wicherts et al. (2011) reported that a higher prevalence of reporting errors were associated with a failure to share data upon request.”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Wicherts et al. (2011)
The cited source says what the claim attributes to it.
In the citing paper
“the current study counts p = .000 as incorrect, whereas this was not the case in for example Wicherts et al. (2011; see also Appendix A).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Wicherts et al. (2011)
The cited source says what the claim attributes to it.
In the citing paper
“Wicherts et al. (2011) included only t, F, and χ 2 values of which the reported p-value was smaller than .05”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Wicherts et al. (2011)
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Statistical analyses of research data are quite error prone [1,2,3], accounts of statistical results may be inaccurate [4], and decisions that researchers make during the analytical phase of a study may lean towards the goal of achieving a preferred (significant) result [5,6,7,8].”
BackgroundCohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
THE JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION
(1911) · Journal of the American Medical Association · doi:10.1001/jama.1911.04260070032017 · Cited by 3,361
Publication Manual of the American Psychological Association
DeCleene, K. E., & Fogo, J. (2012) · Occupational Therapy In Health Care · doi:10.3109/07380577.2011.629024 · Cited by 3,167
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · cited 1×
“This procedure is not without its critics [24,25], but remains to be used extensively in psychology [26] and related fields.”
BackgroundStatistical Methods for Research Workers.
Hotelling, H., & Fisher, R. A. (1930) · Journal of the American Statistical Association · doi:10.2307/2278215 · Cited by 1,529
International Journal of Methods in Psychiatric Research
(2006) · International Journal of Methods in Psychiatric Research · doi:10.1002/(issn)1557-0657 · Cited by 1,492
Publication Manual of the American Psychological Association: BOOK REVIEW
Yao, R. (2011) · Family and Consumer Sciences Research Journal · doi:10.1111/j.1552-3934.2011.02081.x · Cited by 1,451
Cited by
Items 101–200 of 417
Reproducibility in the Social Sciences
Moody, J. W., Keister, L. A., & Ramos, M. C. (2022) · Annual Review of Sociology · doi:10.1146/annurev-soc-090221-035954 · Cited by 67
Full Disclosure: Doing behavioural science necessitates sharing
Peters, G. J., Abraham, C., & Crutzen, R. (2018) · doi:10.31234/osf.io/n7p5m · Cited by 56
Kepes, S., Keener, S. K., McDaniel, M. A., & Hartman, N. S. (2022) · Journal of Organizational Behavior · doi:10.1002/job.2623 · Cited by 71
“…journals' partiality for statistically significant results, preferences for theory development and for “hot” new topics (Campbell & Wilmot, 2018; Kepes & McDaniel, 2013), lack of data transparency requirements (Wicherts et al., 2006, 2011), and shunning of replications (Makel et al., 2012).”
BackgroundHow has your science data grown? Digital curation and the human factor: a critical literature review
Poole, A. H. (2014) · Archives and Museum Informatics · doi:10.1007/s10502-014-9236-y · Cited by 84
“Researchers also need training, especially on appraisal and on data planning (Donnelly and North 2011; Wicherts et al. 2011).”
BackgroundThe Weak Spots in Contemporary Science (and How to Fix Them)
Wicherts, J. M. (2017) · Animals · doi:10.3390/ani7120090 · Cited by 60
“Unfortunately, the common reluctance to share data often precludes independent assessments of the severity of bias due to p-hacking [58].”
BackgroundSelf-citationHybrid Imaging: Instrumentation and Data Processing
Cal-Gonzalez, J., Rausch, I., Shiyam Sundar, L. K., Lassen, M. L., Muzik, O., Moser, E., Papp, L., & Beyer, T. (2018) · Frontiers in Physics · doi:10.3389/fphy.2018.00047 · Cited by 52
“Moreover, many researchers remain reluctant to share their data with an expert audience [304, 305] beyond describing them as part of peer-reviewed publications.”
BackgroundLevenstein, M. C., & Lyle, J. A. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918758319 · Cited by 51
Research Data in Core Journals in Biology, Chemistry, Mathematics, and Physics
Womack, R. P. (2015) · PLOS ONE · doi:10.1371/journal.pone.0143460 · Cited by 50
“At the same time, some may be reluctant to share data, although this could be related to the weakness of the authors’ findings [12].”
Braga, P. H. P., Hébert, K., Hudgins, E. J., Scott, E. R., Edwards, B. P. M., Sánchez‐Reyes, L. L., Grainger, M., Foroughirad, V., Hillemann, F., Binley, A. D., et al. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14108 · Cited by 65
“However, code and data sharing may be met by individual reluctance, temporary embargoes, or partially prevented by privacy and confidentiality reasons (Figueiredo, 2017; Tenopir et al., 2015; Wicherts et al., 2011).”
BackgroundPeters, G.-J., & Crutzen, R. (2024) · Meta-Psychology · doi:10.15626/mp.2022.3638 · Cited by 61
The publication and reproducibility challenges of shared data
Poldrack, R. A., & Poline, J. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.11.008 · Cited by 54
Estimating the Replicability of Sports and Exercise Science Research
Murphy, J., Caldwell, A. R., Mesquida, C., Ladell, A. J. M., Martínez, A. E., Tual, A., Denys, A. T., Cameron, B., Hooren, B. V., Parr, B., et al. (2025) · Sports Medicine · doi:10.1007/s40279-025-02201-w · Cited by 59
“Data concealment can facilitate poor statistical analyses, manipulation, or selective reporting of results and a higher rate of statistical errors [89].”
Liu, D. M., & Salganik, M. J. (2019) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023119849803 · Cited by 47
Fundamental concepts in the Cyclus nuclear fuel cycle simulation framework
Huff, K., Gidden, M., Carlsen, R., Flanagan, R., McGarry, M. B., Opotowsky, A. C., Schneider, E. A., Scopatz, A., & Wilson, P. P. H. (2016) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2016.01.014 · Cited by 53
“Furthermore, openness allows cross-institutional collaboration, increases software robustness [4, 5], improves the strength and quality of results through peer review [6, 7, 8, 9, 10], and cultivates an ecosystem of modeling options.”
BackgroundKrafczyk, M. S., Shi, A., Bhaskar, A., Marinov, D., & Stodden, V. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0069 · Cited by 63
“With recent efforts showing some high profile works failing to reproduce [40–42], attempts have been made to determine why such works fail to reproduce [43,44], what policies can be taken to decrease reproduction failures [36,45] and whether such policies are effective [11,42,46].”
BackgroundReplication initiatives will not salvage the trustworthiness of psychology
Coyne, J. C. (2016) · BMC Psychology · doi:10.1186/s40359-016-0134-3 · Cited by 48
“Requests for sharing of data when sharing is mandated are often rejected or simply ignored, with evidence that authors of studies with exaggerated interpretation of findings or outright errors are less responsive to requests for their data [47].”
BackgroundGordon, M., Viganola, D., Dreber, A., Johannesson, M., & Pfeiffer, T. (2021) · PLOS ONE · doi:10.1371/journal.pone.0248780 · Cited by 44
“There is, however, some debate on the appropriateness of interpreting p-values as a measure of strength of evidence [35, 36].”
BackgroundPerspectives on Open Science and scientific data sharing:an interdisciplinary workshop.
Destro‐Bisol, G., Anagnostou, P., Capocasa, M., Bencivelli, S., Cerroni, A., Contreras, J. L., Enke, N., Fantini, B., Greco, P., Heeney, C., et al. (2014) · PubMed · doi:10.4436/jass.92006 · Cited by 46
“It is also common for statistical analyses of data to be prone to error (Bakker & Wicherts, 2011) and for unwillingness to share data to be associated with the prevalence of errors in papers from which data were requested (Wicherts et al., 2011).”
Self-citationFoundations of plasma standards
Alves, L. L., Becker, M. M., van Dijk, J., Gans, T., Go, D. B., Stapelmann, K., Tennyson, J., Turner, M. M., & Kushner, M. J. (2023) · Plasma Sources Science and Technology · doi:10.1088/1361-6595/acb810 · Cited by 50
Data Submission and Quality in Microarray-Based MicroRNA Profiling
Witwer, K. W. (2013) · Clinical Chemistry · doi:10.1373/clinchem.2012.193813 · Cited by 43
Codifying Collegiality: Recent Developments in Data Sharing Policy in the Life Sciences
Kanter, G. P., Ƶinner, D. E., & Campbell, E. G. (2014) · PLoS ONE · doi:10.1371/journal.pone.0108451 · Cited by 42
“Similar estimates–obtained through surveys, public data searches, and field experiments–have since been reported in various subfields of genetics and other life sciences [5–12].”
ResultVermeulen, I., Beukeboom, C. J., Batenburg, A., Avrămiea, A.-E., Stoyanov, D. S., van de Velde, B., & Oegema, D. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096333 · Cited by 47
“Based on these potential mechanisms, we expect, similar to what was found in psychology literature (Bakker & Wicherts, 2011; Wicherts et al., 2011) that (1) p-values that are “too low” for their accompanying test statistics are reported more often than p-values that are “too high.””
BackgroundMethodResultInfluentialInterpreting t-Statistics Under Publication Bias: Rough Rules of Thumb
Winship, C., & Zhuo, X. (2018) · Journal of Quantitative Criminology · doi:10.1007/s10940-018-9387-8 · Cited by 47
A 20-Year Review of Outcome Reporting Bias in Moderated Multiple Regression
O’Boyle, E. H., Banks, G. C., Carter, K., Walter, S. L., & Yuan, Z. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9539-8 · Cited by 43
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39
Conclusion: Data, open science, and methodological reform in second language acquisition research
Marsden, E. J., & Plonsky, L. (2018) · Language learning and language teaching · doi:10.1075/lllt.51.10mar · Cited by 43
Psychological science’s replicability crisis and what it means for science in the courtroom.
Chin, J. M. (2014) · Psychology Public Policy and Law · doi:10.1037/law0000012 · Cited by 43
A review of the literature on ethical issues related to scientific authorship
Hosseini, M., & Gordijn, B. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1750957 · Cited by 38
“For example, unethical use of statistical methods that leads to favoring/cherry-picking specific aspects of results is seen as a biased interpretation of results (Irvin 2009; Kilkenny et al. 2009; Wicherts, Bakker, and Molenaar 2011; Wang, Yan, and Katz 2018).”
BackgroundResultInfluentialThe OKCupid dataset: A very large public dataset of dating site users
Kirkegaard, E. O. W., & Bjerrekær, J. D. (2016) · Open Differential Psychology · doi:10.26775/odp.2016.11.03 · Cited by 38
“Despite many years of advocacy of proponents, it is still uncommon for social scientists to publicly share their datasets and even sharing data on request is rare (Krawczyk & Reuben, 2012; Savage & Vickers, 2009; Tenopir et al., 2011; Wicherts et al., 2011).”
BackgroundPersonalizing Medicine Through Hybrid Imaging and Medical Big Data Analysis
Papp, L., Spielvogel, C. P., Rausch, I., Hacker, M., & Beyer, T. (2018) · Frontiers in Physics · doi:10.3389/fphy.2018.00051 · Cited by 44
Full disclosure: Doing behavioural science necessitates sharing
Peters, G.-J., Abraham, C., & Crutzen, R. (2012) · PsycEXTRA Dataset · doi:10.1037/e544792013-002 · Cited by 36
Chang, X., Gao, H., & Li, W. (2024) · Journal of Accounting Research · doi:10.1111/1475-679x.12579 · Cited by 37
Resources available for autism research in the big data era: a systematic review
Al‐Jawahiri, R., & Milne, E. (2017) · PeerJ · doi:10.7717/peerj.2880 · Cited by 40
Implications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 38
“(e.g., Wicherts et al., 2011).”
The Validity of the Tool “statcheck” in Discovering Statistical Reporting Inconsistencies
Nuijten, M. B., van Assen, M. A. L. M., Hartgerink, C. H. J., Epskamp, S., & Wicherts, J. M. (2017) · doi:10.31234/osf.io/tcxaj · Cited by 33
Self-citationData sharing in sociology journals
Zenk-Möltgen, W., & Lepthien, G. (2014) · Online Information Review · doi:10.1108/oir-05-2014-0119 · Cited by 33
Do Corresponding Authors Take Responsibility for Their Work? A Covert Survey
Teunis, T., Nota, S. P. F. T., & Schwab, J. H. (2014) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-014-3868-3 · Cited by 35
“unanswered [16, 17].”
Towards Bayesian Model-Based Demography: Agency, Complexity and Uncertainty in Migration Studies
Bijak, J. (2021) · Methodos series · doi:10.1007/978-3-030-83039-7 · Cited by 48
Data sharing in biological anthropology
Mulligan, C. J., Boyer, D., Turner, T. R., Delson, E., & Leonard, W. R. (2022) · American Journal of Biological Anthropology · doi:10.1002/ajpa.24499 · Cited by 35
Repurposing Legacy Data: Innovative Case Studies
Berman, J. J. (2015) · Cited by 33
Analytic Review as a Solution to the Misreporting of Statistical Results in Psychological Science
Sakaluk, J., Williams, A., & Biernat, M. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614549257 · Cited by 32
“Adopting this more purposive approach to selecting analyses or articles for AR would better facilitate addressing the problem of innocent or deliberate misreporting, as authors are most likely to misreport statistics from analyses related to their central claims (Wicherts et al., 2011).”
BackgroundInfluentialAn A–Z of Applied Linguistics Research Methods
Loewen, S., & Plonsky, L. (2016) · doi:10.1007/978-1-137-40322-3_1 · Cited by 86
Science revolves around the data
(2013) · Journal of Open Psychology Data · doi:10.5334/jopd.e1 · Cited by 29
Self-citationAcceso abierto a los datos de investigación, una vía hacia la colaboración científica
Melero, R., & Miguel, H. S. (2014) · Revista española de Documentación Científica · doi:10.3989/redc.2014.4.1154 · Cited by 30
Verify original results through reanalysis before replicating
Nuijten, M. B., Bakker, M., Maassen, E., & Wicherts, J. M. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000791 · Cited by 31
Self-citationCostello, W., & Dorris, E. (2019) · Health Expectations · doi:10.1111/hex.12972 · Cited by 32
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/umq8d · Cited by 37
Self-citationEffects of emotionally oriented parental interventions: a systematic review and meta-analysis
Zahl‐Olsen, R., Severinsen, L., Stiegler, J. R., Fernee, C. R., Simhan, I., Rekdal, S. S., & Bertelsen, T. B. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1159892 · Cited by 34
A Duty to Describe: Better the Devil You Know Than the Devil You Don’t
Brown, S. D., Furrow, D. R., Hill, D. F., Gable, J. C., Porter, L. P., & Jacobs, W. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551749 · Cited by 34
Managing the data commons: Controlled sharing of scholarly data
Eschenfelder, K. R., & Johnson, A. M. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23086 · Cited by 34
Schumm, W. R. (2015) · Comprehensive Psychology · doi:10.2466/17.cp.4.24 · Cited by 32
Best practices for genetic and genomic data archiving
Leigh, D. M., Vandergast, A. G., Hunter, M. E., Crandall, E. D., Funk, W. C., Garroway, C. J., Hoban, S., Oyler‐McCance, S. J., Rellstab, C., Segelbacher, G., et al. (2024) · Nature Ecology & Evolution · doi:10.1038/s41559-024-02423-7 · Cited by 36
Krefeld-Schwalb, A., & Scheibehenne, B. (2022) · Marketing Letters · doi:10.1007/s11002-022-09662-3 · Cited by 31
Wessel, J. A., Danhof, N. A., van EEKELEN, R., Diamond, M. P., Legro, R. S., Peeraer, K., D’Hooghe, T., Erdem, M., Dankert, T., Cohlen, B. J., et al. (2022) · Human Reproduction Update · doi:10.1093/humupd/dmac021 · Cited by 27
“Previous empirical evidence in our research field has demonstrated that results of the RCTs without IPD have lower quality and more methodological issues compared to RCTs who shared IPD and therefore may result in different findings (Wicherts et al., 2011; Bordewijk et al., 2020).”
Kurata, K., Matsubayashi, M., & Mine, S. (2017) · SAGE Open · doi:10.1177/2158244017717301 · Cited by 29
Statistical reporting inconsistencies in experimental philosophy
Colombo, M., Duev, G., Nuijten, M. B., & Sprenger, J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0194360 · Cited by 29
Push button replication: Is impact evaluation evidence for international development verifiable?
Wood, B. D. K., Müller, R., & Brown, A. N. (2018) · PLOS ONE · doi:10.1371/journal.pone.0209416 · Cited by 24
Examining Psychological Science Through Systematic Meta-Method Analysis: A Call for Research
Elson, M. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919863296 · Cited by 27
Research Collaborations and Quality in Research: Foes or Friends?
Vaudano, E. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_293 · Cited by 25
“Reluctance to data sharing has been shown to correlate with poor quality of the research results; see, for example, the paper by Wicherts et al. (Wicherts et al. 2011).”
ResultSchofield, P. N., Kulka, U., Tapio, S., & Grosche, B. (2019) · International Journal of Radiation Biology · doi:10.1080/09553002.2019.1589026 · Cited by 24
“One worrying observation is that data from marginally significant or poorly reported experiments seems to dominate the data sharing deficiency (Wicherts et al. 2011), suggesting that there is concern amongst some authors that their data are not checked or reanalyzed.”
BackgroundAstorga, F., Groom, Q., Shimabukuro, P. H. F., Manguin, S., Noesgaard, D., Orrell, T. M., Sinka, M. E., Hirsch, T., & Schigel, D. (2023) · One Health · doi:10.1016/j.onehlt.2023.100484 · Cited by 23
“This could represent a general disciplinary bias, with medical and clinical disciplines tending to present a lower and unpaired commitment with some data sharing and reusing practices when compared to ecology and environmental sciences [47 – 51].”
ResultRufiange, M., Rousseau‐Blass, F., & Pang, D. (2019) · Veterinary Record Open · doi:10.1136/vetreco-2018-000322 · Cited by 26
“0 per cent of papers (all published in BMC Veterinary Research) included a statement that data were available on contacting the author; however, author compliance to requests for data access may be low.(40) Journal impact factor is calculated from the ratio between citations received (to articles published in the prec…”
BackgroundWagner, E. M., Honer, W. G., Sommer, I. E., Koops, S., Blumberger, D. M., Daskalakis, Z. J., Lange, J. J. D.-D., Bais, L., Knegtering, H., Aleman, A., et al. (2020) · The World Journal of Biological Psychiatry · doi:10.1080/15622975.2020.1733080 · Cited by 26
“…respond to our request and two authors declined to participate – this observation of a reduced likelihood to receive requested data has been frequently reported in the literature (Savage and Vickers 2009; Wicherts et al. 2011) and, thus, is an inherent limitation of all pooled-analyses approaches.”
BackgroundA Balanced Data Archiving Policy for Long-Term Studies
Whitlock, M. C., Bronstein, J. L., Bruna, E. M., Ellison, A. M., Fox, C. W., McPeek, M. A., Moore, A. J., Noor, M. A. F., Rausher, M. D., Rieseberg, L. H., et al. (2015) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2015.12.001 · Cited by 23
“Data archiving allows the scientific community to produce new results economically [6] and permits the essential verification of existing results [7].”
BackgroundForero, D. A., Curioso, W. H., & Patrinos, G. P. (2021) · BMC Research Notes · doi:10.1186/s13104-021-05817-z · Cited by 26
“In addition, in a sample of 49 published articles it was found that the reluctance to share data was associated with a weaker evidence and a higher number of errors in the reporting of statistical results [17].”
BackgroundGauging the Purported Costs of Public Data Archiving for Long-Term Population Studies
Evans, S. R. (2016) · PLoS Biology · doi:10.1371/journal.pbio.1002432 · Cited by 24
Excess success for three related papers on racial bias
Francis, G. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00512 · Cited by 34
“Such p-value misreporting is not uncommon (Bakker and Wicherts, 2011; Wicherts et al., 2011).”
The Database for Reaching Experiments and Models
Walker, B., & Kording, K. P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0078747 · Cited by 23
“Willingness to share data has been shown to be an indicator of good science [7], so we are confident in the quality of these studies.”
BackgroundImker, H. J., Luong, H., Mischo, W. H., Schlembach, M. C., & Wiley, C. (2021) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2021.102369 · Cited by 27
Efendić, E., & van Zyl, L. E. (2019) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v45i0.1607 · Cited by 29
“Wicherts, Bakker, and Molenaar (2011), for example, found that the reluctance to share data is associated with weak statistical evidence against the null hypothesis and errors in the reporting of statistical results.”
BackgroundThe puzzle of sharing scientific data
Priego, L. P., Wareham, J. D., & Romasanta, A. K. S. (2022) · Industry and Innovation · doi:10.1080/13662716.2022.2033178 · Cited by 30
Can we share models if sharing data is not an option?
Li, Z., Mao, F., & Wu, C. (2022) · Patterns · doi:10.1016/j.patter.2022.100603 · Cited by 24
“8,12,13 However, according to a recent survey, most researchers from across nations and disciplines considered that lack of access to research data significantly impedes scientific discoveries.”
BackgroundThe New Reality of Reproducibility: The Role of Data Work in Scientific Research
Feinberg, M., Sutherland, W., Nelson, S. B., Jarrahi, M. H., & Rajasekar, A. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392840 · Cited by 28
“This rising concern with reproducibility, which has been termed a “crisis” in the field of psychology, has led to a reevaluation of certain aspects of scientific practice, such as the publication of research data and materials and the preferential publication of positive results [29, 58].”
BackgroundOpen access meta-analysis for psychotherapy research.
Baldwin, S. A., & Re, A. C. D. (2016) · Journal of Counseling Psychology · doi:10.1037/cou0000091 · Cited by 22
“Indeed in a follow-up study to Wicherts et al. (2006), Wicherts, Bakker, and Molenaar (2011) found a relationship between strength of evidence as reported in a manuscript and reluctance to share data (they limited their analysis to the papers from the Journal of Personality and Social Psychology…”
BackgroundBayeSuites: An open web framework for massive Bayesian networks focused on neuroscience
Michiels, M., Larrañaga, P., & Bielza, C. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.11.066 · Cited by 25
Meursinge Reynders, R., Ladu, L., & Di Girolamo, N. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.07.001 · Cited by 18
Bordewijk, E. M., Wang, R., van Wely, M., Costello, M. F., Norman, R. J., Teede, H., Gurrin, L. C., Mol, B. W., & Li, W. (2020) · Human Reproduction Update · doi:10.1093/humupd/dmaa031 · Cited by 18
IS THERE EPISTEMIC JUSTIFICATION FOR SECRECY IN SCIENCE?
de Ridder, J. (2013) · Episteme · doi:10.1017/epi.2013.18 · Cited by 19
Tsujimoto, Y., Fujii, T., Onishi, A., Omae, K., Luo, Y., Imai, H., Takahashi, S., Itaya, T., Pinson, C., Nevitt, S. J., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.10.004 · Cited by 17
Ciência aberta, gestão de dados de pesquisa e novas possibilidades para a editoração científica
Sales, L. F., Costa, M., & Shintaku, M. (2020) · ABEC eBooks · doi:10.21452/978-85-93910-04-3.cap1 · Cited by 21
Barraviera, B., Brandão, A. A., Correia, D., Ferreira, R. S., & Shintaku, M. (2020) · ABEC eBooks · doi:10.21452/978-85-93910-04-3.cap17 · Cited by 17
A Ciência Aberta no CNPq: acesso aberto aos dados e resultados de pesquisas
Tonini, A. M., Santana, P. H. D. A., & Scherrer, T. M. (2020) · ABEC eBooks · doi:10.21452/978-85-93910-04-3.cap16 · Cited by 17
Nichols, D. M., & Twidale, M. B. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23741 · Cited by 20
Gestão de dados em periódicos científicos
Shintaku, M., & Ferreira, R. S. (2020) · ABEC eBooks · doi:10.21452/978-85-93910-04-3.cap18 · Cited by 19
As potencialidades do data paper na ciência atual
Rocha, L. (2020) · ABEC eBooks · doi:10.21452/978-85-93910-04-3.cap13 · Cited by 16
Care to share? Experimental evidence on code sharing behavior in the social sciences
Krähmer, D., Schächtele, L., & Schneck, A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289380 · Cited by 20
Open Science: A Review of Its Effectiveness and Implications for Service Research
Vaerenbergh, Y. V., Hazée, S., & Zwienenberg, T. J. (2025) · Journal of Service Research · doi:10.1177/10946705251338461 · Cited by 24
“While Wicherts et al. (2011) found a higher prevalence of statistical errors in papers for which the data were not shared, Nuijten et al. (2017) found no relationship between data sharing and reporting inconsistencies across three large retrospective observational studies.”
Veroniki, A. A., Straus, S. E., Ashoor, H. M., Stewart, L. A., Clarke, M. J., & Tricco, A. C. (2016) · Trials · doi:10.1186/s13063-016-1238-z · Cited by 16
“For example, the original study authors may worry that a re-analysis might show an error or a pattern they missed [8], they may have limited publication rights for the data as a study sponsor owns them [9], they may have moved to a different university and lost the data, or the data might be old and saved in an inacce…”
BackgroundMalaysian researchers on open data: The first national survey on awareness, practices and attitudes
Hodonu-Wusu, J. O., Abdullah, N., & Abrizah, A. (2020) · Malaysian Journal of Library & Information Science · doi:10.22452/mjlis.vol25no2.1 · Cited by 21
“…of Open Although open data sharing policies as well as the technology to facilitate data sharing are quite increasing (Crosas 2012; Crosas et al. 2015), scholars do not share their data even when ethically required to do so (Wicherts, Bakker and Molenaar 2011), especially through publications.”
BackgroundSharing is caring: Ethical implications of transparent research in psychology.
Bosma, C. M., & Granger, A. M. (2022) · American Psychologist · doi:10.1037/amp0001002 · Cited by 25
New Explorations into International Relations
Choi, S. (2016) · University of Georgia Press eBooks · doi:10.1353/book45330 · Cited by 15
Szkuta, K., & Osimo, D. (2016) · foresight · doi:10.1108/fs-06-2014-0040 · Cited by 16
“…(2015) recent work show that sharing data is still impeded by lack of formal recognition (citations, co-authorship), other study suggests that willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results (Wicherts et al., 2011).”
Lukács, G., & Specker, E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0240259 · Cited by 16
“This should not only help future efforts as described above but also help the reviewers more accurately assess the quality of the study under review [72, 73].”
BackgroundCurrent reproducibility practices in management: What they are versus what they could be
Cortina, J. M., Köhler, T., & Aulisi, L. C. (2023) · Journal of Management Scientific Reports · doi:10.1177/27550311231202696 · Cited by 16
“As was mentioned earlier, Wicherts et al. (2011) found support for the notion that authors whose evidence in support of hypotheses is fl imsy and or driven by questionable analytic decisions are more reluctant to share their data.”
BackgroundResultReplication is more than hitting the lottery twice
Asendorpf, J. B., Conner, M. T., Fruyt, F. D., Houwer, J. D., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R. M., Nosek, B. A., et al. (2013) · Research portal (Tilburg University) · doi:10.1002/per.1920/abstract · Cited by 14
Self-citationROCZNIKI PSYCHOLOGICZNE / ANNALS OF PSYCHOLOGY
Soroko, E., & Psychologii, I. (2013) · Cited by 14
Jussim, L. J., Krosnick, J. A., Stevens, S. T., & Anglin, S. M. (2019) · Psychologica Belgica · doi:10.5334/pb.496 · Cited by 16
The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs
Supplee, L. H., Ammerman, R. T., DUGGAN, A. K., List, J. A., & Suskind, D. L. (2021) · Prevention Science · doi:10.1007/s11121-021-01322-8 · Cited by 19
“Open data and code shine a light on problems with analysis and reporting of findings, including genuine mistakes (Wicherts et al., 2011).”
BackgroundPreregistration and Registered Reports in Sociology: Strengths, Weaknesses, and Other Considerations
Manago, B. N. (2023) · The American Sociologist · doi:10.1007/s12108-023-09563-6 · Cited by 18
“Second, there is considerable evidence that, on the whole, preregistration, registered reports, and similar open-science practices increase research quality (Chambers and Tzavella 2022; Nuijten et al., 2016; Scheel, 2021; Soderberg et al., 2021; Wicherts et al. 2011).”
BackgroundInfluentialDoes the use of unusual combinations of datasets contribute to greater scientific impact?
Yu, Y., & Romero, D. M. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2402802121 · Cited by 13
Metadata
- Authors
- Wicherts, J. M., Bakker, M., & Molenaar, D.
- Affiliations
- University of Amsterdam (NL)
- Corresponding author
- Wicherts, Jelte M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-11-02
- Volume
- 6
- Issue
- 11
- Pages
- e26828
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3206853
- OpenAlex
- W2072620887
- Cited by
- 424
- References
- 50
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 18 · 2023: 26 · 2022: 23 · 2021: 54 · 2020: 34 · 2019: 27 · 2018: 37 · 2017: 33 · 2016: 40 · 2015: 41 · 2014: 33 · 2013: 27 · 2012: 15
- FWCI
- 69.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- Tilburg University Research Portal (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
- Research portal (Tilburg University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (451-07-016)
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Sociology and Political Science › Data Analysis and Archiving
- Keywords
- research data sharing, data archiving, reanalysis, statistical reporting errors, statistical significance, quality of evidence, null hypothesis, psychology journals, data verification
- MeSH terms
- Cooperative Behavior, Humans, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.