The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Meta‐research: The art of getting it wrong
Abstract
Meta‐analysis has major strengths, but sometimes it can often lead to wrong and misleading answers. In this SRSM presidential address, I discuss some case studies that exemplify these problems, including examples from meta‐analyses of both clinical trials and observational associations. I also discuss issues of effect size estimation, bias (in particular significance‐chasing biases), and credibility in meta‐research. I examine the factors that affect the credibility of meta‐analyses, including magnitude of effects, multiplicity of analyses, scale of data, flexibility of analyses, reporting, and conflicts of interest. Under the current circumstances, a survey of expert meta‐analysts attending the SRSM meeting showed that most of them believe that the true effect is practically equally likely to lie within the 95% confidence interval of a meta‐analysis or outside of it. Finally, I address the placement of meta‐analysis in the wider current research agenda and make a plea for adoption of more prospective meta‐designs. In many/most/all fields, all primary original research may be designed, executed, and interpreted as a prospective meta‐analysis. Copyright © 2011 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 7×
“Many effects tend to diminish over time (Gehr et al., 2006; Ioannidis, 2005a, 2006; Jennions and Moeller, 2002; Krum and Tonkin, 2003; Leimu and Koricheva, 2004; Trikalinos et al., 2004; Zollner and Pritchard, 2007).”
BackgroundResultInfluentialSelf-citationOptimal Medical Therapy with or without PCI for Stable Coronary Disease
Boden, W. E., O'Rourke, R. A., Teo, K. K., Hartigan, P. M., Maron, D. J., Kostuk, W. J., Knudtson, M., Dada, M., Casperson, P., Harris, C. L., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa070829 · Cited by 4,699 · cited 1×
“Our meta-analysis in 2005 (Ioannidis and Katritsis, 2005) found no benefit from invasive management, and a large randomized trial published 2 years later also found no benefit (Boden et al., 2007).”
Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
Genome-wide association studies for complex traits: consensus, uncertainty and challenges
McCarthy, M. I., Abecasis, G. R., Cardon, L. R., Goldstein, D. B., Little, J., Ioannidis, J. P. A., & Hirschhorn, J. N. (2008) · Nature Reviews Genetics · doi:10.1038/nrg2344 · Cited by 2,891
Self-citationPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Self-citationSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationWhy Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Self-citation
Cited by
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Self-citationMeta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
Homocysteine lowering interventions for preventing cardiovascular events
Martí-Carvajal, A. J., Solà, I., Lathyris, D., & Salanti, G. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006612.pub2 · Cited by 520
Szucs, D., & Ioannidis, J. P. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117164 · Cited by 453
Self-citationAchat, D. L., Deleuze, C., Landmann, G., Pousse, N., Ranger, J., & Augusto, L. (2015) · Forest Ecology and Management · doi:10.1016/j.foreco.2015.03.042 · Cited by 353
Metadata
- Authors
- Ioannidis, J. P. A.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2010-07
- Volume
- 1
- Issue
- 3-4
- Pages
- 169-184
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W1972260676
- Cited by
- 227
- References
- 23
- FWCI
- 8.93
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Meta‐research: The art of getting it wrong. https://app.cochat.ai/papers/10.1002/jrsm.19
- The paper
- Ioannidis, J. P. A. (2010). Meta‐research: The art of getting it wrong. Research Synthesis Methods. https://doi.org/10.1002/jrsm.19
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.