Ain’t necessarily so: Review and critique of recent meta-analyses of behavioral medicine interventions in health psychology.
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: We examined four meta-analyses of behavioral interventions for adults (Dixon, Keefe, Scipio, Perri, & Abernethy, 2007; Hoffman, Papas, Chatkoff, & Kerns, 2007; Irwin, Cole, & Nicassio, 2006; and Jacobsen, Donovan, Vadaparampil, & Small, 2007) that have appeared in the Evidence Based Treatment Reviews section of Health Psychology. DESIGN: Narrative review. MAIN OUTCOME MEASURES: We applied the following criteria to each meta-analysis: (1) whether each meta-analysis was described accurately, adequately, and transparently in the article; (2) whether there was an adequate attempt to deal with methodological quality of the original trials; (3) the extent to which the meta-analysis depended on small, underpowered studies; and (4) the extent to which the meta-analysis provided valid and useful evidence-based recommendations. RESULTS: Across the four meta-analyses, we identified substantial problems with the transparency and completeness with which these meta-analyses were reported, as well as a dependence on small, underpowered trials of generally poor quality. CONCLUSION: Results of our exercise raise questions about the clinical validity and utility of the conclusions of these meta-analyses. Results should serve as a wake up call to prospective authors, reviewers, and end-users of meta-analyses now appearing in the literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 1×
“…quality as scored with these scales, some threshold should be set as a basis for outright exclusion of trials that either do not achieve this minimal quality or for which analyses should be conducted both with and without their inclusion (Higgins & Altman, 2008; Juni, Altman, & Egger, 2001).”
BackgroundMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · cited 1×
“While dozens of scales show validity for evaluating methodological quality, they nonetheless differ in their ratings in an arbitrary fashion (Juni, Witschi, Bloch, & Egger, 1999).”
BackgroundWhy Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · cited 3×
“Even in the case of a true null, such a collection of published small studies would be expected to produce a robust effect size estimate (Howard et al., 2009; Ioannidis, 2008a; Kraemer et al., 1998).”
BackgroundResultThe Effect of Group Psychosocial Support on Survival in Metastatic Breast Cancer
Goodwin, P. J., Leszcz, M., Ennis, M., Koopmans, J., Vincent, L., Guther, H., Drysdale, E., Hundleby, M., Chochinov, H. M., Navarro, M., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejmoa011871 · Cited by 891 · cited 1×
“Of the 12 psychological interventions with at least 35 patients per cell, Bordeleau et al. (2003) and Goodwin et al. (2001) reported on the same sample.”
BackgroundMeta-analysis of psychological interventions for chronic low back pain
Hoffman, B. M., Papas, R. K., Chatkoff, D. K., & Kerns, R. D. (2007) · Health Psychology · doi:10.1037/0278-6133.26.1.1 · Cited by 844 · cited 4×
“In the following sections, we review each of the four meta-analyses in question based on this set of criteria, beginning with the meta-analysis by Irwin et al. (2006) and followed by the meta-analyses by Hoffman et al. (2007); Dixon et al. (2007), and Jacobsen et al. (2007).”
BackgroundMethodInfluentialInterpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696 · cited 3×
“Even in the case of a true null, such a collection of published small studies would be expected to produce a robust effect size estimate (Howard et al., 2009; Ioannidis, 2008a; Kraemer et al., 1998).”
BackgroundResultControlled Physical Activity Trials in Cancer Survivors: A Systematic Review and Meta-analysis
Schmitz, K. H., Holtzman, J., Courneya, K. S., Mâsse, L. C., Duval, S., & Kane, R. (2005) · Cancer Epidemiology, Biomarkers & Prevention · doi:10.1158/1055-9965.epi-04-0703 · Cited by 638
Lin, E. H. B., Katon, W., Von Korff, M., Tang, L., Williams, Jr., Kroenke, K., Hunkeler, E., Harpole, L., Hegel, M., Arean, P., et al. (2003) · JAMA · doi:10.1001/jama.290.18.2428 · Cited by 590 · cited 4×
“The largest study included in the meta-analysis was a secondary analysis of a 12-month collaborative care intervention for depressed primary care patients (Lin et al., 2003), which Dixon et al.”
BackgroundResultInfluentialReporting standards for research in psychology: why do we need them? What might they be?
APA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008) · American Psychologist · doi:10.1037/0003-066x.63.9.839 · Cited by 571
Cited by
Sanders, M. R., Kirby, J. N., Tellegen, C. L., & Day, J. J. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.04.003 · Cited by 1,077
Sheeran, P., Harris, P. R., & Epton, T. (2014) · Psychological Bulletin · doi:10.1037/a0033065 · Cited by 954
Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence.
Harkin, B., Webb, T. L., Chang, B. P. I., Prestwich, A., Conner, M., Kellar, I., Benn, Y., & Sheeran, P. (2016) · Psychological Bulletin · doi:10.1037/bul0000025 · Cited by 564
Adult Consequences of Late Adolescent Alcohol Consumption: A Systematic Review of Cohort Studies
McCambridge, J., McAlaney, J., & Rowe, R. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000413 · Cited by 450
Does Spirituality or Religion Positively Affect Mental Health? Meta-analysis of Longitudinal Studies
Garssen, B., Visser, A., & Pool, G. (2021) · The International Journal for the Psychology of Religion · doi:10.1080/10508619.2020.1729570 · Cited by 418
“Sub (4): Some authors argue for leaving out small-sized studies (Coyne, Thombs, & Hagedoorn, 2010).”
Background
Metadata
- Authors
- Coyne, J. C., Thombs, B. D., & Hagedoorn, M.
- Venue
- Health Psychology
- Published
- 2010-03
- Volume
- 29
- Issue
- 2
- Pages
- 107-116
- ISSN
- 0278-6133 (ISSN-L)
- 1930-7810
- OpenAlex
- W2167074667
- Cited by
- 80
- References
- 21
- FWCI
- 7.94
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Ain’t necessarily so: Review and critique of recent meta-analyses of behavioral medicine interventions in health psychology. https://app.cochat.ai/papers/10.1037/a0017633
- The paper
- Coyne, J. C., Thombs, B. D., & Hagedoorn, M. (2010). Ain’t necessarily so: Review and critique of recent meta-analyses of behavioral medicine interventions in health psychology. Health Psychology. https://doi.org/10.1037/a0017633
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.