Any casualties in the clash of randomised and observational evidence?: No—recent comparisons have studied selected questions, but we do need more data
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Randomised controlled trials and observational studies are often seen as mutually exclusive, if not opposing, methods of clinical research. Two recent reports, however, identified clinical questions (19 in one report,1 five in the other2) where both randomised trials and observational methods had been used to evaluate the same question, and performed a head to head comparison of them. In contrast to the belief that randomised controlled trials are more reliable estimators of how much a treatment works, both reports found that observational studies did not overestimate the size of the treatment effect compared with their randomised counterparts. The authors say that the merits of well designed observational studies may need to be re-evaluated: case-control and cohort studies may need to assume more respect in assessing medical therapies and largescale observational databases should be better exploited. 1 2 The first claim flies in the face of half a century of thinking, so are these authors right? The combined results from the two reports indeed show a striking concordance between the estimates obtained with the two research designs. A correlation analysis we performed on their combined databases found that the correlation coefficient between the odds ratio of randomised trials and the odds ratio of observational designs is 0.84 (P<0.001). This represents excellent concordance …
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1994) · New England Journal of Medicine · doi:10.1056/nejm199404143301501 · Cited by 4,737
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
Vitamin E Supplementation and Cardiovascular Events in High-Risk Patients
(2000) · New England Journal of Medicine · doi:10.1056/nejm200001203420302 · Cited by 1,957
Randomized Trials or Observational Tribulations?
Pocock, S. J., & Elbourne, D. R. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422511 · Cited by 550
Cited by
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Self-citationCriteria for evaluating evidence on public health interventions
Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002) · Journal of Epidemiology and Community Health · doi:10.1136/jech.56.2.119 · Cited by 1,042
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Self-citationChallenges in evaluating surgical innovation
Ergina, P. L., Cook, J. A., Blazeby, J. M., Boutron, I., Clavien, P.-A., Reeves, B. C., & Seiler, C. M. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61086-2 · Cited by 653
Randomized Trials or Observational Tribulations?
Pocock, S. J., & Elbourne, D. R. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422511 · Cited by 550
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 463
Self-citationAneurysmal Subarachnoid Hemorrhage: the Last Decade
Neifert, S. N., Chapman, E. K., Martini, M. L., Shuman, W. H., Schupper, A. J., Oermann, E. K., Mocco, J., & Macdonald, R. L. (2021) · Translational Stroke Research · doi:10.1007/s12975-020-00867-0 · Cited by 424
Tiihonen, J., Walhbeck, K., Lönnqvist, J., Klaukka, T., Ioannidis, J. P. A., Volavka, J., & Haukka, J. (2006) · BMJ · doi:10.1136/bmj.38881.382755.2f · Cited by 385
Self-citationQuality of reporting of observational longitudinal research.
Tooth, L. (2005) · American Journal of Epidemiology · doi:10.1093/aje/kwi042 · Cited by 363
Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou, P. N. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050873 · Cited by 229
Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs114 · Cited by 204
Prasad, V., Jorgenson, J., Ioannidis, J. P. A., & Cifu, A. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.11.005 · Cited by 90
Self-citation
Metadata
- Authors
- Ioannidis, J. P. A., Haidich, A.-B., & Lau, J.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2001-04-14
- Volume
- 322
- Issue
- 7291
- Pages
- 879-880
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2062248680
- Cited by
- 107
- References
- 5
- FWCI
- 11.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.