The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results From the RCT DUPLICATE Initiative
Open accessAbstract
Background: Regulators are evaluating the use of noninterventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT DUPLICATE initiative (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) uses a structured process to design RWE studies emulating randomized, controlled trials (RCTs) and compare results. We report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications. Methods: We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion and exclusion criteria, selected primary end points, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 preexposure confounders. All study measures were prospectively defined and protocols registered before hazard ratios and 95% CIs were computed. Success criteria for the primary analysis were prespecified for each replication. Results: Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a hazard ratio estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation. Conclusions: Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. Registration: URL: https://www.clinicaltrials.gov ; Unique identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Empagliflozin, Cardiovascular Outcomes, and Mortality in Type 2 Diabetes
Zinman, B., Wanner, C., Lachin, J. M., Fitchett, D., Bluhmki, E., Hantel, S., Mattheus, M., Devins, T., Johansen, O. E., Woerle, H. J., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1504720 · Cited by 12,315
Canagliflozin and Cardiovascular and Renal Events in Type 2 Diabetes
Neal, B., Perkovic, V., Mahaffey, K. W., de Zeeuw, D., Fulcher, G., Erondu, N., Shaw, W., Law, G., Desai, M., & Matthews, D. R. (2017) · New England Journal of Medicine · doi:10.1056/nejmoa1611925 · Cited by 7,830
Liraglutide and Cardiovascular Outcomes in Type 2 Diabetes
Marso, S. P., Daniels, G. H., Brown-Frandsen, K., Kristensen, P., Mann, J. F. E., Nauck, M. A., Nissen, S. E., Pocock, S., Poulter, N. R., Ravn, L. S., et al. (2016) · New England Journal of Medicine · doi:10.1056/nejmoa1603827 · Cited by 7,629
Ticagrelor versus clopidogrel in patients with acute coronary syndromes
Wallentin, L., Becker, R. C., Budaj, A., Cannon, C. P., Emanuelsson, H., Held, C., Horrow, J., Husted, S., James, S., Katus, H., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0904327 · Cited by 7,117
Prasugrel versus clopidogrel in patients with acute coronary syndromes
Wiviott, S. D., Braunwald, E., McCabe, C. H., Montalescot, G., Ruzyllo, W., Gottlieb, S., Neumann, F.-J., Ardissino, D., De Servi, S., Murphy, S. A., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa0706482 · Cited by 6,790
Dapagliflozin and Cardiovascular Outcomes in Type 2 Diabetes
Wiviott, S. D., Raz, I., Bonaca, M. P., Mosenzon, O., Kato, E. T., Cahn, A., Silverman, M. G., Zelniker, T. A., Kuder, J. F., Murphy, S. A., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1812389 · Cited by 6,382
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Saxagliptin and cardiovascular outcomes in patients with type 2 diabetes mellitus
Scirica, B. M., Bhatt, D. L., Braunwald, E., Steg, P. G., Davidson, J., Hirshberg, B., Ohman, P., Frederich, R., Wiviott, S. D., Hoffman, E. B., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1307684 · Cited by 3,470
Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1.
Hernán, M. A., & Robins, J. M. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kwv254 · Cited by 3,115
Effect of Sitagliptin on Cardiovascular Outcomes in Type 2 Diabetes
Green, J. B., Bethel, M. A., Armstrong, P. W., Buse, J. B., Engel, S. S., Garg, J., Josse, R., Kaufman, K. D., Koglin, J., Korn, S., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1501352 · Cited by 2,577
Real-World Evidence — What Is It and What Can It Tell Us?
Sherman, R. E., Anderson, S. A., Dal Pan, G. J., Gray, G. W., Gross, T., Hunter, N. L., LaVange, L., Marinac-Dabic, D., Marks, P. W., Robb, M. A., et al. (2016) · New England Journal of Medicine · doi:10.1056/nejmsb1609216 · Cited by 2,063
Hernán, M. A., Sauer, B. C., Hernández-Díaz, S., Platt, R., & Shrier, I. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.014 · Cited by 1,097
Rosenstock, J., Perkovic, V., Johansen, O. E., Cooper, M. E., Kahn, S. E., Marx, N., Alexander, J. H., Pencina, M., Toto, R. D., Wanner, C., et al. (2019) · JAMA · doi:10.1001/jama.2018.18269 · Cited by 1,084
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Rosenstock, J., Kahn, S. E., Johansen, O. E., Zinman, B., Espeland, M. A., Woerle, H. J., Pfarr, E., Keller, A., Mattheus, M., Baanstra, D., et al. (2019) · JAMA · doi:10.1001/jama.2019.13772 · Cited by 612
Holm, J. C., Nowicka, P., Farpour-Lambert, N. J., O'Malley, G., Hassapidou, M., Weiss, R., & Baker, J. L. (2014) · Obesity Facts · doi:10.1159/000365773 · Cited by 592
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534
Beyond the intention-to-treat in comparative effectiveness research
Hernán, M. A., & Hernández-Díaz, S. (2012) · Clinical Trials · doi:10.1177/1740774511420743 · Cited by 528
One‐to‐many propensity score matching in cohort studies
Rassen, J. A., Shelat, A. A., Myers, J., Glynn, R. J., Rothman, K. J., & Schneeweiss, S. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3263 · Cited by 510
Self-citationBerger, M. L., Sox, H., Willke, R. J., Brixner, D. L., Eichler, H., Goettsch, W., Madigan, D., Makady, A., Schneeweiss, S., Tarricone, R., et al. (2017) · Value in Health · doi:10.1016/j.jval.2017.08.3019 · Cited by 456
Self-citationBerger, M. L., Sox, H., Willke, R. J., Brixner, D. L., Eichler, H., Goettsch, W., Madigan, D., Makady, A., Schneeweiss, S., Tarricone, R., et al. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4297 · Cited by 344
Self-citationWhen and how can real world data analyses substitute for randomized controlled trials?
Franklin, J. M., & Schneeweiss, S. (2017) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.857 · Cited by 316
Self-citationA basic study design for expedited safety signal evaluation based on electronic healthcare data
Schneeweiss, S. (2010) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1926 · Cited by 255
Self-citationLonjon, G., Boutron, I., Trinquart, L., Ahmad, N., Aim, F., Nizard, R., & Ravaud, P. (2014) · Annals of Surgery · doi:10.1097/sla.0000000000000256 · Cited by 235
Hemkens, L. G., Contopoulos-Ioannidis, D. G., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i493 · Cited by 220
Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs114 · Cited by 204
Transparency and Reproducibility of Observational Cohort Studies Using Large Healthcare Databases.
Wang, S., Verpillat, P., Rassen, J., Patrick, A., Garry, E., & Bartels, D. (2016) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.329 · Cited by 197
Self-citationKim, S. C., Solomon, D. H., Rogers, J. R., Gale, S., Klearman, M., Sarsour, K., & Schneeweiss, S. (2017) · Arthritis & Rheumatology · doi:10.1002/art.40084 · Cited by 195
Self-citationHatswell, A. J., Baio, G., Berlin, J. A., Irs, A., & Freemantle, N. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011666 · Cited by 186
Jarow, J. P., LaVange, L., & Woodcock, J. (2017) · JAMA · doi:10.1001/jama.2017.9991 · Cited by 185
Paradoxical Relations of Drug Treatment with Mortality in Older Persons
Glynn, R. J., Knight, E. L., Levin, R., & Avorn, J. (2001) · Epidemiology · doi:10.1097/00001648-200111000-00017 · Cited by 181
Self-citationEvaluating the Use of Nonrandomized Real‐World Data Analyses for Regulatory Decision Making
Franklin, J. M., Glynn, R. J., Martin, D., & Schneeweiß, S. (2019) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1351 · Cited by 159
Self-citationvan Onzenoort, H. V., Menger, F. E., Neef, C., Verberk, W., Kroon, A., de Leeuw, P., & van der Kuy, P. M. (2011) · HYPERTENSION · doi:10.1161/hypertensionaha.111.171074 · Cited by 136
Selective prescribing led to overestimation of the benefits of lipid-lowering drugs
Glynn, R. J., Schneeweiß, S., WANG, P., LEVIN, R., & Avorn, J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.12.012 · Cited by 126
Self-citationFranklin, J. M., Pawar, A., Martin, D., Glynn, R. J., Levenson, M., Temple, R., & Schneeweiss, S. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1633 · Cited by 119
Self-citationPatorno, E., Schneeweiß, S., Gopalakrishnan, C., Martin, D., & Franklin, J. M. (2019) · Diabetes Care · doi:10.2337/dc19-0069 · Cited by 98
Self-citationPatorno, E., Gopalakrishnan, C., Franklin, J. M., Brodovicz, K. G., Massó-González, E. L., Bartels, D. B., Liu, J., & Schneeweiß, S. (2017) · Diabetes Obesity and Metabolism · doi:10.1111/dom.13184 · Cited by 86
Self-citationCharacteristics and Strength of Evidence of COVID-19 Studies Registered on ClinicalTrials.gov
Pundi, K. N., Perino, A. C., Harrington, R. A., Krumholz, H. M., & Turakhia, M. P. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.2904 · Cited by 70
Lin, K. J., Glynn, R. J., Singer, D. E., Murphy, S. N., Lii, J., & Schneeweiß, S. (2017) · Epidemiology · doi:10.1097/ede.0000000000000794 · Cited by 60
Self-citationBhandari, M., Tornetta, P., Ellis, T. J., Audigé, L., Sprague, S., Kuo, J. C., & Swiontkowski, M. F. (2003) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-003-0559-z · Cited by 58
Franklin, J. M., Glynn, R. J., Suissa, S., et al. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1793 · Cited by 54
Self-citationCaliff, R. M., Hernandez, A. F., & Landray, M. (2020) · JAMA · doi:10.1001/jama.2020.13319 · Cited by 46
Real‐World Evidence of Treatment Effects: The Useful and the Misleading
Schneeweiß, S. (2019) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1405 · Cited by 41
Desai, R. J., Levin, R., Lin, K. J., & Patorno, E. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.120.016906 · Cited by 29
Self-citationA Bias in the Evaluation of Bias Comparing Randomized Trials with Nonexperimental Studies
Franklin, J. M., Dejene, S. Z., Huybrechts, K. F., Wang, S. V., Kulldorff, M., & Rothman, K. J. (2017) · Epidemiologic Methods · doi:10.1515/em-2016-0018 · Cited by 19
Self-citationSlattery, J., & Kurz, X. (2020) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5005 · Cited by 14
Cited by
Bosco, E., Hsueh, L., McConeghy, K. W., Gravenstein, S., & Saade, E. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01440-5 · Cited by 568
Wang, S. V., Schneeweiss, S., Initiative, R.-D., Franklin, J. M., Desai, R. J., Feldman, W., Garry, E. M., Glynn, R. J., Lin, K. J., Paik, J., et al. (2023) · JAMA · doi:10.1001/jama.2023.4221 · Cited by 428
Self-citationMethods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations
Mathur, M. B., & VanderWeele, T. J. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-051920-114020 · Cited by 85
Semaglutide and Tirzepatide in Patients With Heart Failure With Preserved Ejection Fraction
Krüger, N., Schneeweiss, S., Fuse, K., Matseyko, S., Sreedhara, S. K., Hahn, G., Schunkert, H., & Wang, S. V. (2025) · JAMA · doi:10.1001/jama.2025.14092 · Cited by 66
Toews, I., Anglemyer, A., Nyirenda, J. L., Alsaid, D., Balduzzi, S., Grummich, K., Schwingshackl, L., & Bero, L. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub3 · Cited by 60
Metadata
- Authors
- Franklin, J. M., Patorno, E., Desai, R. J., Glynn, R. J., Martin, D., Quinto, K., Pawar, A., Bessette, L. G., Lee, H., Garry, E. M., Gautam, N., & Schneeweiss, S.
- Venue
- Circulation
- Published
- 2021-03-09
- Volume
- 143
- Issue
- 10
- Pages
- 1002-1013
- ISSN
- 0009-7322 (ISSN-L)
- 1524-4539
- OpenAlex
- W3112964578
- Cited by
- 299
- References
- 46
- FWCI
- 29.02
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.