The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey
Open accessAbstract
OBJECTIVE: To assess differences in estimated treatment effects for mortality between observational studies with routinely collected health data (RCD; that are published before trials are available) and subsequent evidence from randomized controlled trials on the same clinical question. DESIGN: Meta-epidemiological survey. DATA SOURCES: PubMed searched up to November 2014. METHODS: Eligible RCD studies were published up to 2010 that used propensity scores to address confounding bias and reported comparative effects of interventions for mortality. The analysis included only RCD studies conducted before any trial was published on the same topic. The direction of treatment effects, confidence intervals, and effect sizes (odds ratios) were compared between RCD studies and randomized controlled trials. The relative odds ratio (that is, the summary odds ratio of trial(s) divided by the RCD study estimate) and the summary relative odds ratio were calculated across all pairs of RCD studies and trials. A summary relative odds ratio greater than one indicates that RCD studies gave more favorable mortality results. RESULTS: The evaluation included 16 eligible RCD studies, and 36 subsequent published randomized controlled trials investigating the same clinical questions (with 17,275 patients and 835 deaths). Trials were published a median of three years after the corresponding RCD study. For five (31%) of the 16 clinical questions, the direction of treatment effects differed between RCD studies and trials. Confidence intervals in nine (56%) RCD studies did not include the RCT effect estimate. Overall, RCD studies showed significantly more favorable mortality estimates by 31% than subsequent trials (summary relative odds ratio 1.31 (95% confidence interval 1.03 to 1.65; I(2)=0%)). CONCLUSIONS: Studies of routinely collected health data could give different answers from subsequent randomized controlled trials on the same clinical questions, and may substantially overestimate treatment effects. Caution is needed to prevent misguided clinical decision making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Self-citationMuch ado about nothing: a comparison of the performance of meta-analytical methods with rare events
Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Russell Localio, A. (2007) · Statistics in Medicine · doi:10.1002/sim.2528 · Cited by 938
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Self-citationEarly Surgery versus Conventional Treatment for Infective Endocarditis
Kang, D.-H., Kim, Y.-J., Kim, S.-H., Sun, B. J., Kim, D.-H., Yun, S.-C., Song, J.-M., Choo, S. J., Chung, C.-H., Song, J.-K., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1112843 · Cited by 902
Reduction in cardiovascular events after vascular surgery with atorvastatin: a randomized trial
Durazzo, A. E. S., Machado, F. S., Ikeoka, D. T., De Bernoche, C., Monachini, M. C., Puech-Leão, P., & Caramelli, B. (2004) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2004.01.004 · Cited by 595
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534
Johnson, M. L., Crown, W., Martin, B. C., Dormuth, C. R., & Siebert, U. (2009) · Value in Health · doi:10.1111/j.1524-4733.2009.00602.x · Cited by 285
Fonarow, G. C., Abraham, W. T., Albert, N. M., Stough, W. G., Gheorghiade, M., Greenberg, B. H., O'Connor, C. M., Sun, J. L., Yancy, C. W., & Young, J. B. (2008) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2008.03.048 · Cited by 257
Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs114 · Cited by 204
Cited by
Why Most Clinical Research Is Not Useful
Ioannidis, J. P. A. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002049 · Cited by 655
Self-citationMa, C., Avenell, A., Bolland, M., Hudson, J., Stewart, F., Robertson, C., Sharma, P., Fraser, C., & MacLennan, G. (2017) · BMJ · doi:10.1136/bmj.j4849 · Cited by 464
Wang, S. V., Schneeweiss, S., Initiative, R.-D., Franklin, J. M., Desai, R. J., Feldman, W., Garry, E. M., Glynn, R. J., Lin, K. J., Paik, J., et al. (2023) · JAMA · doi:10.1001/jama.2023.4221 · Cited by 428
Ochen, Y., Beks, R. B., van Heijl, M., Hietbrink, F., Leenen, L. P. H., van der Velde, D., Heng, M., van der Meijden, O., Groenwold, R. H. H., & Houwert, R. M. (2019) · BMJ · doi:10.1136/bmj.k5120 · Cited by 351
When and how can real world data analyses substitute for randomized controlled trials?
Franklin, J. M., & Schneeweiss, S. (2017) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.857 · Cited by 316
Metadata
- Authors
- Hemkens, L. G., Contopoulos-Ioannidis, D. G., & Ioannidis, J. P. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2016-02-08
- Volume
- 352
- Pages
- i493
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2258482782
- Cited by
- 220
- References
- 12
- FWCI
- 24.14
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. https://app.cochat.ai/papers/10.1136/bmj.i493
- The paper
- Hemkens, L. G., Contopoulos-Ioannidis, D. G., & Ioannidis, J. P. A. (2016). Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ. https://doi.org/10.1136/bmj.i493
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.