Populating the data ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles.
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
The vast majority of scientific articles published to-date have not been accompanied by concomitant publication of the underlying research data upon which they are based. This state of affairs precludes the routine re-use and re-analysis of research data, undermining the efficiency of the scientific enterprise, and compromising the credibility of claims that cannot be independently verified. It may be especially important to make data available for the most influential studies that have provided a foundation for subsequent research and theory development. Therefore, we launched an initiative-the Data Ark-to examine whether we could retrospectively enhance the preservation and accessibility of important scientific data. Here we report the outcome of our efforts to retrieve, preserve, and liberate data from 111 of the most highly-cited articles published in psychology and psychiatry between 2006-2011 (n = 48) and 2014-2016 (n = 63). Most data sets were not made available (76/111, 68%, 95% CI [60, 77]), some were only made available with restrictions (20/111, 18%, 95% CI [10, 27]), and few were made available in a completely unrestricted form (15/111, 14%, 95% CI [5, 22]). Where extant data sharing systems were in place, they usually (17/22, 77%, 95% CI [54, 91]) did not allow unrestricted access. Authors reported several barriers to data sharing, including issues related to data ownership and ethical concerns. The Data Ark initiative could help preserve and liberate important scientific data, surface barriers to data sharing, and advance community discussions on data stewardship.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Authors may raise concerns about participant confidentiality and data security, 51 but many data sets in health collect anonymised data and authors need not commit to openly sharing their data or sharing any variables that include personal information.”
From Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials. (2022) · cites this paper as [51]
The cited source says what the claim attributes to it.
In the citing paper
“However, assessment of analytic reproducibility is typically precluded by poor access to research data [7–14].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“In a recent attempt [8] to obtain data for some of the most highly cited articles in psychology and psychiatry published over the last decade, the majority of datasets were not made available (76/111, 68%).”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Psychological science, in particular, has a poor record on data sharing [8,12,13], and is currently engaged in a field-wide conversation regarding credibility and scientific practices [17].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Two-sided confidence intervals for the single proportion: comparison of seven methods
Newcombe, R. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8<857::aid-sim777>3.0.co;2-e · Cited by 5,107
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Self-citationPromoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
(1973) · Cited by 1,638
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436
Self-citationData Sharing by Scientists: Practices and Perceptions
Tenopir, C., Allard, S., Douglass, K., Aydinoglu, A. U., Wu, L., Read, E., Manoff, M., & Frame, M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0021101 · Cited by 1,432
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760
Self-citationKidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., et al. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002456 · Cited by 700
Self-citationWhy Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563
Self-citation
Cited by
The relationship between burnout, depression, and anxiety: a systematic review and meta-analysis.
Koutsimani, P., Montgomery, A., & Georganta, K. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00284 · Cited by 1,091
“The problems of finding appropriate data to conduct analyses and the reluctance/inability of authors to provide such data when directly contacted has been identified within the literature as a significant barrier to conducting comprehensive meta-analysis (Hardwicke and Ioannidis, 2018).”
BackgroundData sharing practices and data availability upon request differ across scientific disciplines
Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., Pedaste, M., Raju, M., Astapova, A., Lukner, H., et al. (2021) · Scientific Data · doi:10.1038/s41597-021-00981-0 · Cited by 387
Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., et al. (2018) · Royal Society Open Science · doi:10.1098/rsos.180448 · Cited by 336
Self-citationHardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020) · Royal Society Open Science · doi:10.1098/rsos.190806 · Cited by 217
Self-citationHardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620979806 · Cited by 187
Self-citation
Metadata
- Authors
- Hardwicke, T. E., & Ioannidis, J. P. A.
- Affiliations
- Stanford University (US)
- Corresponding author
- Hardwicke, Tom E.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-08-02
- Volume
- 13
- Issue
- 8
- Pages
- e0201856
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6072126
- OpenAlex
- W2885658721
- Cited by
- 78
- References
- 33
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 5 · 2023: 14 · 2022: 13 · 2021: 14 · 2020: 8 · 2019: 8 · 2018: 2
- FWCI
- 12.87
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Sociology and Political Science › Data Analysis and Archiving
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- research data sharing, data preservation, data accessibility, highly cited papers, psychology, psychiatry, barriers to data sharing, data ownership, research ethics, data stewardship
- MeSH terms
- Scholarly Communication, Humans, Ownership, Periodicals as Topic, Psychiatry, Psychology, Publishing, Bibliometrics, Information Dissemination, Datasets as Topic, Data Curation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Populating the data ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. https://app.cochat.ai/papers/10.1371/journal.pone.0201856
- The paper
- Hardwicke, T. E., & Ioannidis, J. P. A. (2018). Populating the data ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PLOS ONE. https://doi.org/10.1371/journal.pone.0201856
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.