ODDPub – a Text-Mining Algorithm to Detect Data Sharing in Biomedical Publications
Open accessPublished version of a preprint
Abstract
Abstract Open research data are increasingly recognized as a quality indicator and an important resource to increase transparency, robustness and collaboration in science. However, no standardized way of reporting Open Data in publications exists, making it difficult to find shared datasets and assess the prevalence of Open Data in an automated fashion. We developed ODDPub (Open Data Detection in Publications), a text-mining algorithm that screens biomedical publications and detects cases of Open Data. Using English-language original research publications from a single biomedical research institution (n=8689) and randomly selected from PubMed (n=1500) we iteratively developed a set of derived keyword categories. ODDPub can detect data sharing through field-specific repositories, general-purpose repositories or the supplement. Additionally, it can detect shared analysis code (Open Code). To validate ODDPub, we manually screened 792 publications randomly selected from PubMed. On this validation dataset, our algorithm detected Open Data publications with a sensitivity of 0.74 and specificity of 0.97. Open Data was detected for 11.5% (n=91) of publications. Open Code was detected for 1.4% (n=11) of publications with a sensitivity of 0.73 and specificity of 1.00. We compared our results to the linked datasets found in the databases PubMed and Web of Science. Our algorithm can automatically screen large numbers of publications for Open Data. It can thus be used to assess Open Data sharing rates on the level of subject areas, journals, or institutions. It can also identify individual Open Data publications in a larger publication corpus. ODDPub is published as an R package on GitHub.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CARE emphasizes that data governance must not be neutral or extractive, but rather benefit Indigenous communities and be shaped by their authority and ethical traditions.
“CARE emphasizes that data governance must not be neutral or extractive, but rather benefit Indigenous communities and be shaped by their authority and ethical traditions.”
From Preventing AI extractivism: the case for braiding indigenous data justice with ABS for stronger AI data governance (2026) · cites this paper as Carroll et al. 2020
The Definition for Open Data & Open Code (p. 4) concerns criteria for open biomedical research data, and the Outlook (p. 16) concerns detecting data sharing in publications. Searches for CARE, Indigenous communities, and Indigenous data governance found no passage addressing community benefit, authority, or ethical traditions. (LLM, conf=0.76)
Experimental — accuracy not yet measuredOCAP and the CARE Principles can be braided inside the ABS triad of consent, terms, and benefit-sharing to create a digital register model for AI data governance.
“we translate those rules into a digital register by braiding two Indigenous data-governance frameworks-OCAP ® (Ownership, Control, Access, Possession) and the CARE Principles (Collective Benefit, Authority to Control, Responsibility, Ethics)-inside the ABS triad of consent, terms, and benefit-sharing.”
From Preventing AI extractivism: the case for braiding indigenous data justice with ABS for stronger AI data governance (2026) · cites this paper as Carroll et al. 2020
The Discussion (p. 14) describes ODDPub as an algorithm for detecting Open Data sharing in biomedical publications, and the Outlook (p. 16) discusses dataset citations and registry metadata. Searches for OCAP, the CARE Principles, ABS benefit-sharing, and an AI-governance digital register found no passage addressing how those frameworks could be combined. (LLM, conf=0.76)
Experimental — accuracy not yet measuredCARE elevates the rationale for OCAP, contextualizing operational practices within an ethical worldview grounded in self-determination and respect.
“In this sense, CARE elevates the rationale for OCAP ® , contextualizing operational practices within an ethical worldview grounded in self-determination and respect (Carroll et al. 2020).”
From Preventing AI extractivism: the case for braiding indigenous data justice with ABS for stronger AI data governance (2026) · cites this paper as Carroll et al. 2020
The Definition for Open Data & Open Code (p. 4) discusses FAIR criteria, while the Outlook (p. 16) discusses detecting biomedical data sharing. Neither addresses CARE, OCAP, or their ethical relationship; searches for those terms and self-determination found no relevant passage in our copy. The printed citation names Carroll et al. (2020), but that is not evidence from the assigned source paper. (LLM, conf=0.78)
Experimental — accuracy not yet measuredThe CARE Principles (Collective Benefit, Authority to Control, Responsibility, Ethics) for Indigenous Data Governance were articulated by the Global Indigenous Data Alliance.
“Movements like the Global Indigenous Data Alliance have articulated principles to guide this, notably the CARE Principles (Collective Benefit, Authority to Control, Responsibility, Ethics) for Indigenous Data Governance (Carroll et al. 2020).”
From Preventing AI extractivism: the case for braiding indigenous data justice with ABS for stronger AI data governance (2026) · cites this paper as Carroll et al. 2020
The Introduction (p. 2) discusses open-data declarations, and the definition section (p. 4) discusses FAIR criteria, but neither addresses Indigenous data governance. Searches for the CARE Principles and the Global Indigenous Data Alliance found no passage in our copy supporting the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · see citing works
The citation advantage of linking publications to research data
Colavizza, G., Hrynaszkiewicz, I., Staden, I., Whitaker, K., & McGillivray, B. (2020) · PLOS ONE · doi:10.1371/journal.pone.0230416 · Cited by 335 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Cousijn, Helena, Amye Kenall, Emma Ganley, Melissa Harrison, David Kernohan, Thomas Lemberger, Fiona Murphy, et al. 2018. 'A Data Citation Roadmap for Scientific Publishers.' Scientific Data 5 (1): 180259. https://doi.org/10.1038/sdata.2018.259. · DOI
- Fecher, Benedikt, Sascha Friesike, and Marcel Hebing. 2015. 'What Drives Academic Data Sharing?' PLOS ONE 10 (2): e0118053. https://doi.org/10.1371/journal.pone.0118053. Federer, Lisa M., Christopher W. Belter, Douglas J. Joubert, Alicia Livinski, Ya-Ling Lu, Lissa N. Snyders, and Holly Thompson. 2018. 'Data Sharing in PLOS ONE: An Analysis of Data Availability Statements.' PLOS ONE 13 (5): e0194768. https://doi.org/10.1371/journal.pone.0194768. Guedj, D, and C Ramjoué. 2015. 'European Commission Policy on Open-Access to Scientific Publications and Research Data in Horizon 2020.' Biomedical Data Journal 01 (1): 11-14. https://doi.org/10.11610/bmdj.01102. Kip, Miriam, Evgeny Bobrov, Nico Riedel, Heike Scheithauer, Thomas Gazlig, and Ulrich Dirnagl. 2019. 'Einführung von Open Data Als Zusätzlicher Indikator Für Die Leistungsorientierte Mittelvergabe (LOM) Forschung an Der Charité - Universitätsmedizin Berlin,' October. https://doi.org/10.5281/ZENODO.3511191. Martone, M, ed. 2014. 'Data Citation Synthesis Group: Joint Declaration of Data Citation Principles.' San Diego CA: FORCE11. https://doi.org/10.25490/a97f-egyk. Milham, Michael P., R. Cameron Craddock, Jake J. Son, Michael Fleischmann, Jon Clucas, Helen Xu, Bonhwang Koo, et al. 2018. 'Assessment of the Impact of Shared Brain Imaging Data on the Scientific Literature.' Nature Communications 9 (1): 2818. https://doi.org/10.1038/s41467-01804976-1. Mullen, Lincoln A., Kenneth Benoit, Os Keyes, Dmitry Selivanov, and Jeffrey Arnold. 2018. 'Fast, Consistent Tokenization of Natural Language Text.' Journal of Open Source Software 3 (23): 655. https://doi.org/10.21105/joss.00655. Naudet, Florian, Charlotte Sakarovitch, Perrine Janiaud, Ioana Cristea, Daniele Fanelli, David Moher, and John P A Ioannidis. 2018. 'Data Sharing and Reanalysis of Randomized Controlled Trials in Leading Biomedical Journals with a Full Data Sharing Policy: Survey of Studies Published in The BMJ and PLOS Medicine .' BMJ , February, k400. https://doi.org/10.1136/bmj.k400. NIH. 2003. 'NIH Data Sharing Policy and Implementation Guidance.' https://grants.nih.gov/grants/policy/data\_sharing/data\_sharing\_guidance.htm. Open Knowledge Foundation. 2017. 'Open Definition 2.1 - Open Definition - Defining Open in Open Data, Open Content and Open Knowledge.' https://opendefinition.org/od/2.1/en/. Piwowar, Heather A., and Todd J. Vision. 2013. 'Data Reuse and the Open Data Citation Advantage.' PeerJ 1 (October): e175. https://doi.org/10.7717/peerj.175. Poppler (version 0.67.0). 2018. https://poppler.freedesktop.org/. Pronk, Tessa E. 2019. 'The Time Efficiency Gain in Sharing and Reuse of Research Data.' Data Science Journal 18 (March): 10. https://doi.org/10.5334/dsj-2019-010. Rowhani-Farid, Anisa, and Adrian G Barnett. 2016. 'Has Open Data Arrived at the British Medical Journal (BMJ) ? An Observational Study.' BMJ Open 6 (10): e011784. https://doi.org/10.1136/bmjopen-2016-011784. Savage, Caroline J., and Andrew J. Vickers. 2009. 'Empirical Study of Data Sharing by Authors Publishing in PLoS Journals.' PLoS ONE 4 (9): e7078. https://doi.org/10.1371/journal.pone.0007078. · DOI
- Stodden, Victoria, Peixuan Guo, and Zhaokun Ma. 2013. 'Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals.' PLoS ONE 8 (6): e67111. https://doi.org/10.1371/journal.pone.0067111. · DOI · Paper page
- Strech, Daniel, Tracey Weissgerber, Ulrich Dirnagl, and on behalf of QUEST Group. 2020. 'Improving the Trustworthiness, Usefulness, and Ethics of Biomedical Research through an Innovative and Comprehensive Institutional Initiative.' PLOS Biology 18 (2): e3000576. https://doi.org/10.1371/journal.pbio.3000576. · DOI
- Vines, Timothy H., Arianne Y.K. Albert, Rose L. Andrew, Florence Débarre, Dan G. Bock, Michelle T. Franklin, Kimberly J. Gilbert, Jean-Sébastien Moore, Sébastien Renaut, and Diana J. Rennison. 2014. 'The Availability of Research Data Declines Rapidly with Article Age.' Current Biology 24 (1): 94-97. https://doi.org/10.1016/j.cub.2013.11.014. · DOI · Paper page
Cited by
Sorted by citations
Assessment of transparency indicators across the biomedical literature: How open is open?
Serghiou, S., Contopoulos-Ioannidis, D. G., Boyack, K. W., Riedel, N., Wallach, J. D., & Ioannidis, J. P. A. (2021) · PLOS Biology · doi:10.1371/journal.pbio.3001107 · Cited by 139 · see citing works
Self-citationIs the future of peer review automated?
Schulz, R., Barnett, A., Bernard, R., Brown, N. J. L., Byrne, J. A., Eckmann, P., Gazda, M. A., Kilicoglu, H., Prager, E. M., Salholz-Hillel, M., et al. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06080-6 · Cited by 80 · see citing works
Hamilton, D. G., Hong, K., Fraser, H., Rowhani-Farid, A., Fidler, F., & Page, M. J. (2023) · BMJ · doi:10.1136/bmj-2023-075767 · Cited by 86 · see citing works
Weissgerber, T., Riedel, N., Kilicoglu, H., Labbé, C., Eckmann, P., ter Riet, G., Byrne, J., Cabanac, G., Capes-Davis, A., Favier, B., et al. (2021) · Nature Medicine · doi:10.1038/s41591-020-01203-7 · Cited by 57 · see citing works
Self-citationDental Research Data Availability and Quality According to the FAIR Principles
Uribe, S. E., Sofi-Mahmudi, A., Raittio, E., Maldupa, I., & Vilne, B. (2022) · Journal of Dental Research · doi:10.1177/00220345221101321 · Cited by 27 · see citing works
Lin, T.-L., Chen, W.-C., Hsiao, T.-F., Liu, H.-I., Yeh, Y.-H., Chan, Y.-K., Lien, W.-S., Kuo, P.-Y., Yu, P. S., & Shuai, H.-H. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.259 · Cited by 6
Li, W., Liu, X., Zhang, Q., Shi, L., Zhang, J.-X., Zhang, X., Luan, J., Li, Y., Xu, T., Zhang, R., et al. (2026) · Accountability in Research · doi:10.1080/08989621.2025.2481943 · Cited by 1 · see citing works
Schulz, M., & Loewen-Colón, J. (2026) · AI & SOCIETY · doi:10.1007/s00146-026-02931-z · Cited by 1 · see citing works
- Authors
- Riedel, N., Kip, M., & Bobrov, E.
- Affiliations
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE)
- Corresponding author
- Riedel, Nico
- Venue
- Data Science Journal
- Publisher
- Ubiquity Press, Ltd.
- Published
- 2020-10-29
- Volume
- 19
- Issue
- 1
- Pages
- 42
- ISSN
- 1683-1470 (ISSN-L)
- OpenAlex
- W3097085767
- Cited by
- 44
- References
- 2
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 7 · 2023: 7 · 2022: 8 · 2021: 3
- FWCI
- 4.23
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Data Science Journal (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Lincoln (University of Nebraska) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Data Science Journal (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- open data, biomedical literature, text mining, data sharing, open code, research data repositories, R package
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
