Best practices in the real-world data life cycle
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
With increasing digitization of healthcare, real-world data (RWD) are available in greater quantity and scope than ever before. Since the 2016 United States 21st Century Cures Act, innovations in the RWD life cycle have taken tremendous strides forward, largely driven by demand for regulatory-grade real-world evidence from the biopharmaceutical sector. However, use cases for RWD continue to grow in number, moving beyond drug development, to population health and direct clinical applications pertinent to payors, providers, and health systems. Effective RWD utilization requires disparate data sources to be turned into high-quality datasets. To harness the potential of RWD for emerging use cases, providers and organizations must accelerate life cycle improvements that support this process. We build on examples obtained from the academic literature and author experience of data curation practices across a diverse range of sectors to describe a standardized RWD life cycle containing key steps in production of useful data for analysis and insights. We delineate best practices that will add value to current data pipelines. Seven themes are highlighted that ensure sustainability and scalability for RWD life cycles: data standards adherence, tailored quality assurance, data entry incentivization, deploying natural language processing, data platform solutions, RWD governance, and ensuring equity and representation in data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Privacy in the age of medical big data
Price, W. N., & Cohen, I. G. (2019) · Nature Medicine · doi:10.1038/s41591-018-0272-7 · Cited by 1,522
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Artificial Intelligence Transforms the Future of Health Care
Noorbakhsh-Sabet, N., Zand, R., Zhang, Y., & Abedi, V. (2019) · The American Journal of Medicine · doi:10.1016/j.amjmed.2019.01.017 · Cited by 626 · cited 1×
“There is now increasing RWD use by other stakeholders: Payors, providers, health systems, and academic institutes can leverage RWD for artificial intelligence (AI)-assisted clinical decision-making [10,11], clinical operations management [12], and population health [13].”
BackgroundDiversity in clinical and biomedical research: a promise yet to be fulfilled
Oh, S. S., Galanter, J., Thakur, N., Pino-Yanes, M., Barcelo, N. E., White, M. J., de Bruin, D. M., Greenblatt, R. M., Bibbins-Domingo, K., Wu, A. H. B., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001918 · Cited by 585 · cited 1×
“However, RWD studies for informing clinical practice tend to be conducted in demographically restricted groups in high-income countries [77].”
BackgroundNatural language processing of social media as screening for suicide risk
Coppersmith, G., Leary, R., Crutchley, P., & Fine, A. (2018) · Biomedical Informatics Insights · doi:10.1177/1178222618792860 · Cited by 506
Why digital medicine depends on interoperability
Lehne, M., Sass, J., Essenwanger, A., Schepers, J., & Thun, S. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0158-1 · Cited by 501
Questions for artificial intelligence in health care
Maddox, T. M., Rumsfeld, J. S., & Payne, P. R. O. (2019) · JAMA · doi:10.1001/jama.2018.18932 · Cited by 385
Inan, O. T., Tenaerts, P., Prindiville, S. A., Reynolds, H. R., Dizon, D. S., Cooper-Arnold, K., Turakhia, M., Pletcher, M. J., Preston, K. L., Krumholz, H. M., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0302-y · Cited by 329
Combining structured and unstructured data for predictive models: a deep learning approach
Zhang, D., Yin, C., Zeng, J., Yuan, X., & Zhang, P. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01297-6 · Cited by 269 · cited 1×
“Unstructured data contain critical context on the patient journey and have remarkable impact on the performance of models [55] and accumulation of pharmaceutical RWE [20], with recent emphasis on necessity for regulatory grade accuracy [56].”
BackgroundReport of the AMIA EHR-2020 Task Force on the status and future direction of EHRs
Payne, T. H., Corley, S., Cullen, T. A., Gandhi, T. K., Harrington, L., Kuperman, G. J., Mattison, J. E., McCallie, D. P., McDonald, C. J., Tang, P. C., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv066 · Cited by 220 · cited 1×
“Incentivization of high-quality data entry is difficult [51]—there is recognition that RWD consumers derive more direct benefit from curation than the person entering or capturing data.”
Background
Cited by
Canali, S., Schiaffonati, V., & Aliverti, A. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000104 · Cited by 378
Moving towards vertically integrated artificial intelligence development
Zhang, J., Budhdeo, S., William, W., Cerrato, P., Shuaib, H., Sood, H., Ashrafian, H., Halamka, J., & Teo, J. T. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00690-x · Cited by 54
Self-citationHospital-wide natural language processing summarising the health data of 1 million patients
Bean, D. M., Kraljevic, Z., Shek, A., Teo, J., & Dobson, R. J. B. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000218 · Cited by 22
Self-citationData Quality–Driven Improvement in Health Care: Systematic Literature Review
Lighterness, A., Adcock, M., Scanlon, L. A., & Price, G. (2024) · Journal of Medical Internet Research · doi:10.2196/57615 · Cited by 26
“Despite its potential, EHR-RWD, and by extension RWE, grapple with formidable barriers, and chief among these is data quality (DQ) [1,2,9,10,13,14].”
Abdalzaher, M. S., Krichen, M., Shaaban, M. F., & Fouda, M. M. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3613669 · Cited by 22
“The lifecycle of IoT data involves several stages, including data collection, storage, processing, analysis, and eventually, archiving or destruction [92].”
Metadata
- Authors
- Zhang, J., Symons, J., Agapow, P., Teo, J. T., Paxton, C. A., Abdi, J., Mattie, H., Davie, C., Torres, A. Z., Folarin, A., Sood, H., Celi, L. A., Halamka, J., Eapen, S., & Budhdeo, S.
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-01-18
- Volume
- 1
- Issue
- 1
- Pages
- e0000003
- ISSN
- 2767-3170 (ISSN-L)
- OpenAlex
- W4205348219
- Cited by
- 67
- References
- 15
- FWCI
- 10.21
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.