The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Small studies are more heterogeneous than large ones: a meta-meta-analysis
Open accessAbstract
Objectives Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). Study design and setting We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. Results Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τS(2) was larger than the large-study τL(2) [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τS was larger than of τL: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. Conclusion Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“but this is also influenced by unique patient characteristics, treatment techniques, and variability in measured outcomes.58”
From The Effect of Upper Cervical Mobilization/Manipulation on Temporomandibular Joint Pain, Maximal Mouth Opening, and Pressure Pain Thresholds: A Systematic Review and Meta-Analysis (2023) · cites this paper as [58]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is due to the wide variety of study designs, the wide confidence intervals of individual studies, and the inclusion of low-weight studies, which can have a disproportionate effect on heterogeneity, especially in random-effects models [58].”
From Safety outcomes of statin vs non-statin lipid-lowering interventions in patients with prior statin-associated muscle symptoms: A systematic review and meta-analysis (2025) · cites this paper as [58]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“[11] argued that underpowered studies are prone to several analytical and reporting biases.”
BackgroundSelf-citationWinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,397
Cited by
Sorted by citations
Items 301–314 of 314
Hadi, M., Saha, S., Petrie, D., Woode, M. E., & Gerdtham, U. (2026) · Drug and Alcohol Review · doi:10.1111/dar.70089
Kao, P.-J., Chien, L., Szu, L., Chang, W.-J., Okoli, C. T. C., & Liu, Y. C. (2026) · Journal of the American Association of Nurse Practitioners · doi:10.1097/jxx.0000000000001251
Multivariate Location-Scale Models for Meta-Analysis
Jansen, K., & Nestler, S. (2026) · Multivariate Behavioral Research · doi:10.1080/00273171.2026.2643868
Trkulja, V. (2026) · medRxiv · doi:10.64898/2026.04.01.26349972
Clinical research on homeopathic preparations: protocol template for a series of systematic reviews
Loef, M., van Haselen, R. A., & Baumgartner, S. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03168-z
Lee, W., Kim, C.-H., Son, J., Yang, K.-Y., Bang, H. T., & Jeon, J.-H. (2026) · Korean Journal of Poultry Science · doi:10.5536/kjps.2026.53.1.49
Crisóstomo, V., Báez-Díaz, C., Sánchez‐Peralta, L. F., Vázquez-López, F., Torrescusa, A., Gálvez‐Montón, C., & Sanchez-Margallo, F. M. (2026) · European journal of medical research · doi:10.1186/s40001-026-04557-9
Liu, J.-C., Yu, H., Chen, Z., & Deng, Q. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1852047
Toro-Huamanchumo, C. J., Rivera-Santillán, A. M., Quispe, C. Z. V., Venancio-Huerta, J. E., Gendrau-Castillo, L. J., Peralta, V., Castro-Reyes, M. M., & Zavala-Loayza, J. A. (2026) · Clinical Medicine Insights Cardiology · doi:10.1177/11795468261433692
Wang, B., Sun, M., Ma, C., & Whittaker, P. (2026) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2026.112428
Slykerman, R. F., Davies, N., Frampton, C., Fong, B., Kassaa, I. A., Dekker, J., & Bassett, S. (2026) · Nutritional Neuroscience · doi:10.1080/1028415x.2026.2729710
Azagew, A. W., Abate, H. K., Mekonnen, C. K., Mekonnen, H. S., Tezera, Z. B., & Belay, G. J. (2024) · ASRIC Journal on Health Sciences · doi:10.68327/wlfx9040
Wang, E., Wei, X., Dua, Y., Perera, M., & Grummet, J. (2026) · British Journal of Urology · doi:10.1111/bju.70471
Afzal, A., & Jumani, A. (2026) · Journal of Clinical Sleep Medicine · doi:10.1007/s44470-026-00220-6
Metadata
- Authors
- IntHout, J., Ioannidis, J. P. A., Borm, G. F., & Goeman, J. J.
- Affiliations
- Radboud University Medical Center (NL)
- Stanford Medicine (US)
- Corresponding author
- IntHout, Joanna
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2015-08
- Volume
- 68
- Issue
- 8
- Pages
- 860-869
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W1973158091
- Cited by
- 314
- References
- 42
- Citations by year
- 2026: 33 · 2025: 37 · 2024: 40 · 2023: 36 · 2022: 38 · 2021: 37 · 2020: 26 · 2019: 23 · 2018: 21 · 2017: 10 · 2016: 7 · 2015: 4
- FWCI
- 2.33
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- study heterogeneity, small-study effects, trial size, random effects models, meta-analysis, sensitivity analysis, meta-meta-analysis
- MeSH terms
- Bayes Theorem, Clinical Trials as Topic, Epidemiologic Methods, Humans, Models, Theoretical, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
