Examining the Use of Real‐World Evidence in the Regulatory Process
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
The 21st Century Cures Act passed by the United States Congress mandates the US Food and Drug Administration to develop guidance to evaluate the use of real‐world evidence (RWE) to support the regulatory process. RWE has generated important medical discoveries, especially in areas where traditional clinical trials would be unethical or infeasible. However, RWE suffers from several issues that hinder its ability to provide proof of treatment efficacy at a level comparable to randomized controlled trials. In this review article, we summarized the advantages and limitations of RWE, identified the key opportunities for RWE, and pointed the way forward to maximize the potential of RWE for regulatory purposes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61602-x · Cited by 21,320
Osterberg, L., & Blaschke, T. (2005) · New England Journal of Medicine · doi:10.1056/nejmra050100 · Cited by 7,974
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974
Decision aids for people facing health treatment or screening decisions
Stacey, D., Légaré, F., Lewis, K., Barry, M. J., Bennett, C. L., Eden, K. B., Holmes-Rovner, M., Llewellyn-Thomas, H., Lyddiatt, A., Thomson, R., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001431.pub5 · Cited by 6,685
Evaluating the quality of medical care
DONABEDIAN, A. (2005) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2005.00397.x · Cited by 6,102
Benchimol, E. I., Smeeth, L., Guttmann, A., Harron, K., Moher, D., Petersen, I., Sørensen, H. T., von Elm, E., Langan, S. M., & RECORD Working Committee (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001885 · Cited by 5,749
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
A New Initiative on Precision Medicine
Collins, F. S., & Varmus, H. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1500523 · Cited by 5,202
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
Self-citationRobins, J. (1986) · Mathematical Modelling · doi:10.1016/0270-0255(86)90088-6 · Cited by 2,451
Estrogen plus Progestin and the Risk of Coronary Heart Disease
Manson, J. E., Hsia, J., Johnson, K. C., Rossouw, J. E., Assaf, A. R., Lasser, N. L., Trevisan, M., Black, H. R., Heckbert, S. R., Detrano, R., et al. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa030808 · Cited by 2,065
Surrogate endpoints in clinical trials: Definition and operational criteria
Prentice, R. L. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080407 · Cited by 2,057
The Path to Personalized Medicine
Hamburg, M. A., & Collins, F. S. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1006304 · Cited by 1,970
Impact of Medication Adherence on Hospitalization Risk and Healthcare Cost
Sokol, M. C., McGuigan, K. A., Verbrugge, R. R., & Epstein, R. S. (2005) · Medical Care · doi:10.1097/01.mlr.0000163641.86870.af · Cited by 1,729
Why we need observational studies to evaluate the effectiveness of health care
Black, N. (1996) · BMJ · doi:10.1136/bmj.312.7040.1215 · Cited by 1,598 · cited 1×
“We, therefore, narrowly examine three types of studies that have been proposed as ways to perform these comparative effectiveness analyses of therapies from RWD: (i) Virtual Comparative Effectiveness Studies ascertain outcomes in both an intervention and control group from an RWD source.(48,49) (ii) Studies using Hist…”
MethodNegative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181d61eeb · Cited by 1,583
Medication Adherence: Its Importance in Cardiovascular Outcomes
Ho, P. M., Bryson, C. L., & Rumsfeld, J. S. (2009) · Circulation · doi:10.1161/circulationaha.108.768986 · Cited by 1,488
Ford, I., & Norrie, J. (2016) · New England Journal of Medicine · doi:10.1056/nejmra1510059 · Cited by 1,405
Major congenital malformations after first-trimester exposure to ACE inhibitors
Cooper, W. O., Hernandez-Diaz, S., Arbogast, P. G., Dudley, J. A., Dyer, S., Gideon, P. S., Hall, K., & Ray, W. A. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa055202 · Cited by 1,192
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156 · cited 1×
“For example, cohort studies using RWD suffer from unmeasured confounding, whereas the use of instrumental variables relies on instrumental assumptions.(81) Because these two study designs require different sets of assumptions, we can estimate the extent of assumption violation and its impact on the risk estimates.”
BackgroundTriangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129
Data-driven prediction of drug effects and interactions
Tatonetti, N. P., Ye, P. P., Daneshjou, R., & Altman, R. B. (2012) · Science Translational Medicine · doi:10.1126/scitranslmed.3003377 · Cited by 936
Self-citationRandomized versus historical controls for clinical trials
Sacks, H., Chalmers, T. C., & Smith, H. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90815-4 · Cited by 834
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754
Off-label prescribing among office-based physicians
Radley, D. C., Finkelstein, S. N., & Stafford, R. S. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.9.1021 · Cited by 711 · cited 1×
“In rare cases, there may be significant off-label usage, but the reasoning behind this off-label usage should be carefully considered in terms of its impacts on patient selection.(71,72) Significant differences between the patients receiving the off-label treatment vs.”
BackgroundReal-world evidence and real-world data for evaluating drug safety and effectiveness.
Corrigan-Curay, J., Sacks, L., & Woodcock, J. (2018) · JAMA · doi:10.1001/jama.2018.10136 · Cited by 471
Key cost drivers of pharmaceutical clinical trials in the United States
Sertkaya, A., Wong, H.-H., Jessup, A., & Beleche, T. (2016) · Clinical Trials · doi:10.1177/1740774515625964 · Cited by 401
How much do clinical trials cost?
Martin, L., Hutchens, M., Hawkins, C., & Radnov, A. (2017) · Nature Reviews Drug Discovery · doi:10.1038/nrd.2017.70 · Cited by 262
Ference, B. A., Julius, S., Mahajan, N., Levy, P. D., Williams, K. A., & Flack, J. M. (2014) · Hypertension · doi:10.1161/hypertensionaha.113.02734 · Cited by 77
Roberts, K., Boland, M. R., Pruinelli, L., Dcruz, J., Berry, A., Georgsson, M., Hazen, R., Sarmiento, R. F., Backonja, U., Yu, K.-H., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw103 · Cited by 25
Self-citation
Cited by
Advancing healthcare AI governance through a comprehensive maturity model based on systematic review
Hussein, R., Zink, A., Ramadan, B., Howard, F. M., Hightower, M., Shah, S., & Beaulieu-Jones, B. K. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02418-7 · Cited by 96
Self-citationHarnessing Real-World Evidence to Advance Cancer Research
Tang, M., Pearson, S.-A., Simes, R. J., & Chua, B. H. (2023) · Current Oncology · doi:10.3390/curroncol30020143 · Cited by 88
Metadata
- Authors
- Beaulieu‐Jones, B. K., Finlayson, S. G., Yuan, W., Altman, R. B., Kohane, I. S., Prasad, V., & Yu, K.
- Venue
- Clinical Pharmacology & Therapeutics
- Publisher
- Wiley
- Published
- 2019-11-14
- Volume
- 107
- Issue
- 4
- Pages
- 843-852
- ISSN
- 0009-9236 (ISSN-L)
- 1532-6535
- OpenAlex
- W2976369321
- Cited by
- 166
- References
- 33
- FWCI
- 20.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.