The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Real-World Evidence — What Is It and What Can It Tell Us?
Abstract
The FDA is developing guidance on the use of “real-world evidence” — health care information from atypical sources, including electronic health records, billing databases, and product and disease registries — to assess the safety and effectiveness of drugs and devices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Personalized nutrition by prediction of glycemic responses
Zeevi, D., Korem, T., Zmora, N., Israeli, D., Rothschild, D., Weinberger, A., Ben-Yacov, O., Lador, D., Avnit-Sagi, T., Lotan-Pompan, M., et al. (2015) · Cell · doi:10.1016/j.cell.2015.11.001 · Cited by 2,929
Salathé, M., Bengtsson, L., Bodnar, T. J., Brewer, D. D., Brownstein, J. S., Buckee, C., Campbell, E. M., Cattuto, C., Khandelwal, S., Mabry, P. L., et al. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002616 · Cited by 500
Booth, C. M., & Tannock, I. F. (2014) · British Journal of Cancer · doi:10.1038/bjc.2013.725 · Cited by 469
Large-scale randomized evidence: Large, simple trials and overviews of trials
Peto, R., Collins, R., & Gray, R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00150-o · Cited by 365
The False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321
Exploring the ethical and regulatory issues in pragmatic clinical trials
Califf, R. M., & Sugarman, J. (2015) · Clinical Trials · doi:10.1177/1740774515598334 · Cited by 191
Self-citationValidation of the Instant Blood Pressure Smartphone App
Plante, T. B., Urrea, B., MacFarlane, Z. T., Blumenthal, R. S., Miller, E. R., Appel, L. J., & Martin, S. S. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0157 · Cited by 159
"A calculated risk": the Salk polio vaccine field trials of 1954
Meldrum, M. L. (1998) · BMJ · doi:10.1136/bmj.317.7167.1233 · Cited by 130
Need for a National Evaluation System for Health Technology
Shuren, J. E., & Califf, R. M. (2016) · JAMA · doi:10.1001/jama.2016.8708 · Cited by 125
Self-citation
Cited by
Pragmatism as a Research Paradigm and Its Implications for Social Work Research
Kaushik, V., & Walsh, C. A. (2019) · Social Sciences · doi:10.3390/socsci8090255 · Cited by 896
Kosiborod, M., Cavender, M. A., Fu, A. Z., Wilding, J. P., Khunti, K., Holl, R. W., Norhammar, A., Birkeland, K. I., Jørgensen, M. E., Thuresson, M., et al. (2017) · Circulation · doi:10.1161/circulationaha.117.029190 · Cited by 813
Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician
Blonde, L., Khunti, K., Harris, S. B., Meizinger, C., & Skolnik, N. S. (2018) · Advances in Therapy · doi:10.1007/s12325-018-0805-y · Cited by 714
Artificial intelligence in COVID-19 drug repurposing
Zhou, Y., Wang, F., Tang, J., Nussinov, R., & Cheng, F. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30192-8 · Cited by 636
Bousquet, J., Schünemann, H. J., Togias, A., Bachert, C., Erhola, M., Hellings, P. W., Klimek, L., Pfaar, O., Wallace, D., Ansotegui, I., et al. (2020) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2019.06.049 · Cited by 609
Network-based approach to prediction and population-based validation of in silico drug repurposing
Cheng, F., Desai, R. J., Handy, D. E., Wang, R., Schneeweiss, S., Barabási, A.-L., & Loscalzo, J. (2018) · Nature Communications · doi:10.1038/s41467-018-05116-5 · Cited by 552
Artificial intelligence and machine learning in clinical development: a translational perspective
Shah, P., Kendall, F., Khozin, S., Goosen, R., Hu, J., Laramie, J., Ringel, M., & Schork, N. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0148-3 · Cited by 498
Leichsenring, F., Steinert, C., Rabung, S., & Ioannidis, J. P. A. (2022) · World Psychiatry · doi:10.1002/wps.20941 · Cited by 485
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467
Berger, M. L., Sox, H., Willke, R. J., Brixner, D. L., Eichler, H., Goettsch, W., Madigan, D., Makady, A., Schneeweiss, S., Tarricone, R., et al. (2017) · Value in Health · doi:10.1016/j.jval.2017.08.3019 · Cited by 456
Berger, M. L., Sox, H., Willke, R. J., Brixner, D. L., Eichler, H., Goettsch, W., Madigan, D., Makady, A., Schneeweiss, S., Tarricone, R., et al. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4297 · Cited by 344
Causal machine learning for predicting treatment outcomes
Feuerriegel, S., Frauen, D., Melnychuk, V., Schweisthal, J., Hess, K., Curth, A., Bauer, S., Kilbertus, N., Kohane, I. S., & van der Schaar, M. (2024) · Nature Medicine · doi:10.1038/s41591-024-02902-1 · Cited by 343
Real-world data: towards achieving the achievable in cancer care
Booth, C. M., Karim, S., & Mackillop, W. J. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0167-7 · Cited by 327
When and how can real world data analyses substitute for randomized controlled trials?
Franklin, J. M., & Schneeweiss, S. (2017) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.857 · Cited by 316
From Data to Causes I: Building A General Cross-Lagged Panel Model (GCLM)
Zyphur, M. J., Allison, P. D., Tay, L., Voelkle, M. C., Preacher, K. J., Zhang, Z., Hamaker, E. L., Shamsollahi, A., Pierides, D. C., Koval, P., et al. (2020) · Organizational Research Methods · doi:10.1177/1094428119847278 · Cited by 306
Franklin, J. M., Patorno, E., Desai, R. J., Glynn, R. J., Martin, D., Quinto, K., Pawar, A., Bessette, L. G., Lee, H., Garry, E. M., et al. (2021) · Circulation · doi:10.1161/circulationaha.120.051718 · Cited by 299
Linde, C., Bongiorni, M. G., Birgersdotter-Green, U., Curtis, A. B., Deisenhofer, I., Furokawa, T., Gillis, A. M., Haugaa, K. H., Lip, G. Y. H., Van Gelder, I., et al. (2018) · EP Europace · doi:10.1093/europace/euy067 · Cited by 296
Biran, N., Ip, A., Ahn, J., Go, R. C., Wang, S., Mathura, S., Sinclaire, B. A., Bednarz, U., Marafelias, M., Hansen, E., et al. (2020) · The Lancet Rheumatology · doi:10.1016/s2665-9913(20)30277-0 · Cited by 275
Association of SARS-CoV-2 Seropositive Antibody Test With Risk of Future Infection
Harvey, R. A., Rassen, J. A., Kabelac, C. A., Turenne, W., Leonard, S., Klesh, R., Meyer, W. A., Kaufman, H. W., Anderson, S., Cohen, O., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.0366 · Cited by 272
Self-citationMehta, H., Tufail, A., Daien, V., Lee, A. Y., Nguyen, V., Ozturk, M., Barthelmes, D., & Gillies, M. C. (2018) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2017.12.002 · Cited by 265
Big data and black-box medical algorithms
Price, W. N. (2018) · Science Translational Medicine · doi:10.1126/scitranslmed.aao5333 · Cited by 254
Atasoy, H., Greenwood, B. N., & McCullough, J. S. (2019) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040218-044206 · Cited by 256
Model‐Informed Drug Development: Current US Regulatory Practice and Future Considerations
Wang, Y., Zhu, H., Madabushi, R., Liu, Q., Huang, S., & Zineh, I. (2019) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1363 · Cited by 228
Persson, F., Nyström, T., Jørgensen, M. E., Carstensen, B., Gulseth, H. L., Thuresson, M., Fenici, P., Nathanson, D., Eriksson, J. W., Norhammar, A., et al. (2018) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.13077 · Cited by 212
Artificial intelligence in oncology: path to implementation
Chua, I. S., Gaziel‐Yablowitz, M., Korach, Z. T., Kehl, K. L., Levitan, N. A., Arriaga, Y. E., Jackson, G. P., Bates, D. W., & Hassett, M. (2021) · Cancer Medicine · doi:10.1002/cam4.3935 · Cited by 179
“79 At their core, data-driven AI methods (e.g., ML and deep learning) recognize the patterns in the training data, and when presented with real-world data, such methods will propagate any bias already present in the data.”
BackgroundMethodAveritt, A. J., Weng, C., Ryan, P., & Perotte, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0277-8 · Cited by 137
Zhou, Q., Chen, Z.-H., Cao, Y.-H., & Peng, S. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00524-2 · Cited by 126
Might microdosing psychedelics be safe and beneficial? An initial exploration
Fadiman, J., & Korb, S. (2019) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2019.1593561 · Cited by 125
Beyond Randomized Clinical Trials: Use of External Controls
Schmidli, H., Häring, D. A., Thomas, M., Cassidy, A., Weber, S., & Bretz, F. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1723 · Cited by 125
Franklin, J. M., Pawar, A., Martin, D., Glynn, R. J., Levenson, M., Temple, R., & Schneeweiss, S. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1633 · Cited by 119
Self-citationToth, P. P., Granowitz, C., Hull, M., Liassou, D., Anderson, A., & Philip, S. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.118.008740 · Cited by 117
Qualls, L. G., Phillips, T. A., Hammill, B. G., Topping, J., Louzao, D. M., Brown, J. S., Curtis, L. H., & Marsolo, K. (2018) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.199 · Cited by 116
“Distributed research networks (DRNs) are critical components of the strategic roadmaps for the National Institutes of Health and the Food and Drug Administration as they work to move toward large-scale systems of evidence generation [1, 2].”
BackgroundPack, A. I., Magalang, U. J., Singh, B., Kuna, S. T., Keenan, B. T., & Maislin, G. (2021) · Sleep · doi:10.1093/sleep/zsaa229 · Cited by 114
Li, Q., Yang, X., Xu, J., Guo, Y., He, X., Hu, H., Lyu, T., Marra, D., Miller, A., Smith, G., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.12967 · Cited by 113
“The absolute SHAP value of each feature indicates its importance.”
BackgroundAdvancing pharmaceutical research: A comprehensive review of cutting-edge tools and technologies
Chen, J., Lin, A., & Luo, P. (2024) · Current Pharmaceutical Analysis · doi:10.1016/j.cpan.2024.11.001 · Cited by 112
Nichols, G. A., Philip, S., Reynolds, K., Granowitz, C. B., & Fazio, S. (2018) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2018-00470 · Cited by 102
Development and validation of early warning score system: A systematic literature review
Fu, L.-H., Schwartz, J., Moy, A., Knaplund, C., Kang, M.-J., Schnock, K. O., Garcia, J. P., Jia, H., Dykes, P. C., Cato, K., et al. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103410 · Cited by 102
New “21st Century Cures” Legislation: Speed and Ease vs Science
Kesselheim, A. S., & Avorn, J. (2017) · JAMA · doi:10.1001/jama.2016.20640 · Cited by 93
Bian, J., Lyu, T., Loiacono, A., Viramontes, T. M., Lipori, G., Guo, Y., Wu, Y., Prosperi, M., George, T. J., Harle, C. A., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa245 · Cited by 92
Natural Language Processing to Ascertain Cancer Outcomes from Medical Oncologist Notes
Kehl, K. L., Xu, W., Lepisto, E., Elmarakeby, H., Hassett, M. J., Van Allen, E. M., Johnson, B. E., & Schrag, D. (2020) · JCO Clinical Cancer Informatics · doi:10.1200/cci.20.00020 · Cited by 90
Harnessing Real-World Evidence to Advance Cancer Research
Tang, M., Pearson, S.-A., Simes, R. J., & Chua, B. H. (2023) · Current Oncology · doi:10.3390/curroncol30020143 · Cited by 88
Halmos, B., Tan, E.-H., Soo, R. A., Cadranel, J., Lee, M. K., Foucher, P., Hsia, T.-C., Hochmair, M., Griesinger, F., Hida, T., et al. (2019) · Lung Cancer · doi:10.1016/j.lungcan.2018.10.028 · Cited by 74
A scoping review of fair machine learning techniques when using real-world data
Huang, Y., Guo, J., Chen, W.-H., Lin, H.-Y., Tang, H., Wang, F., Xu, H., & Bian, J. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104622 · Cited by 61
Rethinking clinical trials for medical AI with dynamic deployments of adaptive systems
Rosenthal, J. T., Beecy, A., & Sabuncu, M. R. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01674-3 · Cited by 59
“Patientoutcomemetrics,althoughmost important,mayalsobethemostdif fi cult to collect, necessitating patient follow-up, data integration and automated abstraction from health records, and such real-worldevidencehasknownlimitations 55 .”
on behalf of the Scottish Diabetes Research Network Epidemiology Group, McGurnaghan, S. J., Brierley, L., Caparrotta, T. M., McKeigue, P. M., Blackbourn, L. A. K., Wild, S. H., Leese, G. P., McCrimmon, R. J., McKnight, J. A., et al. (2019) · Diabetologia · doi:10.1007/s00125-018-4806-9 · Cited by 46
Brown, R. E., Bech, P. G., & Aronson, R. (2020) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.14117 · Cited by 42
Patel, D., Ananthakrishnan, A., Lin, T., Channa, R., Liu, T. Y. A., & Wolf, R. M. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11237120 · Cited by 43
Cardiovascular Safety During and After Use of Phentermine and Topiramate
Ritchey, M. E., Harding, A., Hunter, S., Peterson, C., Sager, P. T., Kowey, P. R., Nguyen, L., Thomas, S., Cainzos-Achirica, M., Rothman, K. J., et al. (2019) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2018-01010 · Cited by 42
Klonoff, D. C., Gutierrez, A., Fleming, A., & Kerr, D. (2019) · Journal of Diabetes Science and Technology · doi:10.1177/1932296819839996 · Cited by 41
Tan, R. S. Y. C., Lin, Q., Low, G. H., Lin, R., Goh, T. C., Chang, C. C. E., Lee, F. F., Chan, W. Y., Tan, W. C., Tey, H. J., et al. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad133 · Cited by 41
Leveraging generative AI for clinical evidence synthesis needs to ensure trustworthiness
Zhang, G., Jin, Q., Jered McInerney, D., Chen, Y., Wang, F., Cole, C. L., Yang, Q., Wang, Y., Malin, B. A., Peleg, M., et al. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104640 · Cited by 40
Pragmatic (trial) informatics: a perspective from the NIH Health Care Systems Research Collaboratory
Richesson, R. L., Green, B. B., Laws, R., Puro, J., Kahn, M. G., Bauck, A., Smerek, M., Van Eaton, E. G., Zozus, M., Ed Hammond, W., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx016 · Cited by 39
Minozzi, S., Cinquini, M., Gianola, S., Castellini, G., Gerardi, C., & Banzi, R. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.04.001 · Cited by 35
Self-fulfilling prophecies and machine learning in resuscitation science
De-Arteaga, M., & Elmer, J. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2022.10.014 · Cited by 28
Sacrocolpopexy compared with transvaginal mesh surgery: A systematic review and meta-analysis
Zhang, C., Sun, Z., Yang, J., Xu, T., Zhu, L., & Lang, J. (2021) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16324 · Cited by 26
The use of real-world data in drug repurposing
Park, K. (2021) · Translational and Clinical Pharmacology · doi:10.12793/tcp.2021.29.e18 · Cited by 22
“The information generated from RWD provides important real-world evidence (RWE) to inform patient care, safety surveillance, therapeutic development, outcomes research, and comparative effectiveness studies [8].”
BackgroundRetraining reflexes: clinical translation of spinal reflex operant conditioning
Eftekhar, A., Norton, J. J. S., McDonough, C. M., & Wolpaw, J. R. (2018) · Neurotherapeutics · doi:10.1007/s13311-018-0643-2 · Cited by 20
Temporal relationship of computed and structured diagnoses in electronic health record data
Schulz, W. L., Young, H. P., Coppi, A., Mortazavi, B. J., Lin, Z., Jean, R. A., & Krumholz, H. M. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01416-x · Cited by 20
Pragmatic Comparative Effectiveness Trials and Learning Health Systems in Pain Medicine
Salmasi, V., Terkawi, A. S., & Mackey, S. C. (2023) · Anesthesiology Clinics · doi:10.1016/j.anclin.2023.03.010 · Cited by 16
Practice-Based Research Networks and the Mandate for Real-World Evidence
Hall-Lipsy, E., Barraza, L., & Robertson, C. (2018) · American Journal of Law & Medicine · doi:10.1177/0098858818789428 · Cited by 16
Representing and utilizing clinical textual data for real world studies: An OHDSI approach
Keloth, V. K., Banda, J. M., Gurley, M., Heider, P. M., Kennedy, G., Liu, H., Liu, F., Miller, T., Natarajan, K., V Patterson, O., et al. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104343 · Cited by 16
Treat to target in weight management with semaglutide: Real‐world evidence from an eHealth clinic
Seier, S., Stamp‐Larsen, K., Jensen, S. B. K., Torekov, S. S., & Gudbergsen, H. (2025) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.70096 · Cited by 15
The Randomized Registry Trial: Two Birds, One Stone
Liu, J. B., D’Angelica, M. I., & Ko, C. Y. (2017) · Annals of Surgery · doi:10.1097/sla.0000000000002166 · Cited by 15
A Fair Individualized Polysocial Risk Score for Identifying Increased Social Risk in Type 2 Diabetes
Huang, Y., Guo, J., Donahoo, W. T., Lee, Y. A., Fan, Z., Lu, Y., Chen, W.-H., Tang, H., Bilello, L., Saguil, A. A., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-52960-9 · Cited by 14
Blancas, I., Olier, C., Conde, V., Bayo, J. L., Herrero, C., Zarcos-Pedrinaci, I., Carabantes, F., Baena-Cañada, J. M., Cruz, J., & Ruiz-Borrego, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-83622-1 · Cited by 12
Shao, H., Thorpe, L. E., Islam, S., Bian, J., Guo, Y., Li, P., Bost, S., Dabelea, D., Conway, R., Crume, T., et al. (2025) · Diabetes Care · doi:10.2337/dc24-1972 · Cited by 8
Generative Medical Event Models Improve with Scale
Waxler, S., Blazek, P., White, D., Sneider, D., Chung, K., Nagarathnam, M., Williams, P., Voeller, H., Wong, K., Swanhorst, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.12104 · Cited by 4
Metadata
- Authors
- Sherman, R. E., Anderson, S. A., Dal Pan, G. J., Gray, G. W., Gross, T., Hunter, N. L., LaVange, L., Marinac-Dabic, D., Marks, P. W., Robb, M. A., Shuren, J., Temple, R., Woodcock, J., Yue, L. Q., & Califf, R. M.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2016-12-08
- Volume
- 375
- Issue
- 23
- Pages
- 2293-2297
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2560688864
- Cited by
- 2,063
- References
- 15
- FWCI
- 253.84
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.