The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses
Open accessAbstract
Evidence on how much medical interventions work may change over time. It is important to determine what fluctuations in the treatment effect reported by randomized trials and their metaanalyses may be expected and whether extreme fluctuations signal future major changes. We applied recursive cumulative metaanalysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 interventions in two medical fields (pregnancy/perinatal medicine, n = 45 interventions; myocardial infarction, n = 15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes greater than 2.5 standard deviations of the expected. With 500 accumulated patients, the pooled odds ratio may change by 0.6- to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74- and 1.35-fold for pregnancy/perinatal medicine and between 0.83- and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of preeclampsia), where recent mega-trials have contradicted prior metaanalyses, as well as in four other examples where early large treatment effects were dissipated when more data appeared. Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their metaanalyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may sometimes signal further major changes in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Self-citationCumulative Meta-Analysis of Therapeutic Trials for Myocardial Infarction
Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992) · New England Journal of Medicine · doi:10.1056/nejm199207233270406 · Cited by 1,140
Self-citationTrial of Calcium to Prevent Preeclampsia
Levine, R. J., Hauth, J. C., Curet, L. B., Sibai, B. M., Catalano, P. M., Morris, C. D., DerSimonian, R., Esterlitz, J. R., Raymond, E. G., Bild, D. E., et al. (1997) · New England Journal of Medicine · doi:10.1056/nejm199707103370201 · Cited by 634
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.4.281 · Cited by 632
Self-citationLarge Trials vs Meta-analysis of Smaller Trials: How Do Their Results Compare?
Cappelleri, J. C. (1996) · JAMA · doi:10.1001/jama.1996.03540160054033 · Cited by 336
Understanding research synthesis (meta-analysis).
Mosteller, F., & Colditz, G. A. (1996) · Annual Review of Public Health · doi:10.1146/annurev.pu.17.050196.000245 · Cited by 228
Cited by
Replication validity of genetic association studies
Ioannidis, J. P. A., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001) · Nature Genetics · doi:10.1038/ng749 · Cited by 1,832
Self-citationThe case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Self-citationAmstad, F. T., Meier, L. L., Fasel, U., Elfering, A., & Semmer, N. K. (2011) · Journal of Occupational Health Psychology · doi:10.1037/a0022170 · Cited by 1,654
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationNeoadjuvant Versus Adjuvant Systemic Treatment in Breast Cancer: A Meta-Analysis
Mauri, D., Pavlidis, N., & Ioannidis, J. P. A. (2005) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dji021 · Cited by 1,200
Self-citationDwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127
Effect of obesity on survival of women with breast cancer: systematic review and meta-analysis
Protani, M., Coory, M., & Martin, J. H. (2010) · Breast Cancer Research and Treatment · doi:10.1007/s10549-010-0990-0 · Cited by 962
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751
“There is an increasing interest in appraising biases in large-scale across many topics [38,39].”
BackgroundSelf-citationIoannidis, J. P. A. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081086 · Cited by 716
Self-citationInterpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696
Self-citationLiving Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap
Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001603 · Cited by 596
Genetic associations in large versus small studies: an empirical assessment
Ioannidis, J. P., Trikalinos, T. A., Ntzani, E. E., & Contopoulos-Ioannidis, D. G. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12516-0 · Cited by 579
Self-citationBalk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478
Juul, A. B., Wetterslev, J., Gluud, C., Kofoed-Enevoldsen, A., Jensen, G., Callesen, T., N⊘rgaard, P., Fruergaard, K., Bestle, M., Vedelsdal, R., et al. (2006) · BMJ · doi:10.1136/bmj.332.7556.1482 · Cited by 359
Nykänen, H., & Koricheva, J. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.12768.x · Cited by 350
Ioannidis, J. P. A., & Trikalinos, T. A. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.019 · Cited by 325
Self-citationThorlund, K., Imberger, G., Walsh, M., Chu, R., Gluud, C., Wetterslev, J., Guyatt, G., Devereaux, P. J., & Thabane, L. (2011) · PLoS ONE · doi:10.1371/journal.pone.0025491 · Cited by 319
Genetic associations: false or true?
Ioannidis, J. P. A. (2003) · Trends in Molecular Medicine · doi:10.1016/s1471-4914(03)00030-3 · Cited by 289
Self-citationMauri, D., Pavlidis, N., Polyzos, N. P., & Ioannidis, J. P. A. (2006) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djj357 · Cited by 286
Self-citationUCHL1 is a Parkinson's disease susceptibility gene
Maraganore, D. M., Lesnick, T. G., Elbaz, A., Chartier‐Harlin, M., Gasser, T., Krüger, R., Hattori, N., Mellick, G. D., Quattrone, A., Satoh, J., et al. (2004) · Annals of Neurology · doi:10.1002/ana.20017 · Cited by 262
Self-citationKLEIN, J. B., JACOBS, R. H., & REINECKE, M. A. (2007) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/chi.0b013e3180592aaa · Cited by 254
Sample size and power considerations in network meta-analysis
Thorlund, K., & Mills, E. J. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-41 · Cited by 197
Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time
Trikalinos, T. A., Churchill, R., Ferri, M., Leucht, S., Tuunainen, A., Wahlbeck, K., & Ioannidis, J. P. A. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.02.018 · Cited by 177
Self-citationConceptualizing changes in behavior in intervention research: the range of possible changes model
De Los Reyes, A., & Kazdin, A. E. (2006) · Psychological Review · doi:10.1037/0033-295x.113.3.554 · Cited by 172
Gong, Y., Huang, Z., Christensen, E., & Gluud, C. (2007) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2007.01235.x · Cited by 131
Early stopping of randomized clinical trials for overt efficacy is problematic.
Bassler, D., Montori, V. M., Briel, M., Glasziou, P., & Guyatt, G. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.07.016 · Cited by 130
Evolution and Translation of Research Findings: From Bench to Where
Ioannidis, J. P. A. (2006) · PLoS Clinical Trials · doi:10.1371/journal.pctr.0010036 · Cited by 129
Self-citationNakaoka, H., & Inoue, I. (2009) · Journal of Human Genetics · doi:10.1038/jhg.2009.95 · Cited by 111
Kambeitz, J. P., & Howes, O. D. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.07.034 · Cited by 96
Poor replication validity of biomedical association studies reported by newspapers
Dumas-Mallet, E., Smith, A., Boraud, T., & Gonon, F. (2017) · PLOS ONE · doi:10.1371/journal.pone.0172650 · Cited by 76
Early goal-directed therapy for severe sepsis and septic shock: A living systematic review
Simpson, S. Q., Gaines, M., Hussein, Y., & Badgett, R. G. (2016) · Journal of Critical Care · doi:10.1016/j.jcrc.2016.06.017 · Cited by 55
How to critically read ecological meta-analyses
Lortie, C. J., Stewart, G., Rothstein, H., & Lau, J. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1109 · Cited by 39
“Cumulatively adding single studies does little to increase the capacity of a meta-analysis to detect treatment effects (Ioannidis et al., 1998, Ioannidis and Lau, 2001), but a major strength of meta-analysis is nonetheless that many small or even low-quality studies can be combined to provide a…”
BackgroundSelf-citationCatalá-López, F., Hutton, B., Page, M. J., Ridao, M., Driver, J. A., Alonso-Arroyo, A., Forés-Martos, J., Macías Saint-Gerons, D., Vieta, E., Valencia, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0581-9 · Cited by 11
“To further assess the consistency of evidence over time, we will conduct cumulative meta-analyses in the order of publication year [33, 34].”
Method
Metadata
- Authors
- Ioannidis, J. P. A.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2001-01-23
- Volume
- 98
- Issue
- 3
- Pages
- 831-836
- ISSN
- 0027-8424 (ISSN-L)
- OpenAlex
- W2023598977
- Cited by
- 192
- References
- 7
- FWCI
- 16.96
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.