Testing continuous moderators in meta‐analysis: A comparison of procedures
The DOI matches its Crossref record.
Abstract
Using Monte Carlo simulation, we compare the statistical power and Type I error rates of three procedures commonly applied in meta‐analysis to detect the influence of a continuous moderator variable on the estimated effect sizes from a set of independent primary studies. We use the conventional T test (TTEST), the Hedges & Olkin (1985) z HO test (H&O), and the Rosenthal & Rubin (1982) z RR test (R&R). The H&O and R&R procedures weight every estimated effect size by its inverse‐variance, whereas in the conventional T test the individual effect sizes are unweighted. Results show an adequate control of the Type I error rate for the three procedures in all of the conditions. The statistical power is very similar in the H&O and R&R procedures, and systematically lower in the conventional T test. Both H&O and R&R procedures were similarly the most appropriate and powerful to detect a quantitative moderator variable on effect sizes, supporting the suitability of applying weighting schemes in meta‐analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A meta-analytic review of work–family conflict and its antecedents
Byron, K. (2005) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2004.08.009 · Cited by 2,336
Ego depletion and the strength model of self-control: a meta-analysis
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010) · Psychological Bulletin · doi:10.1037/a0019486 · Cited by 2,265
Snyder, H. R. (2013) · Psychological Bulletin · doi:10.1037/a0028727 · Cited by 1,667
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267
Deficits in behavioural inhibition in substance abuse and addiction: a meta-analysis.
Smith, J. L., Mattick, R. P., Jamadar, S. D., & Iredale, J. M. (2014) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2014.08.009 · Cited by 546
Byron, K., Khazanchi, S., & Nazarian, D. (2010) · Journal of Applied Psychology · doi:10.1037/a0017868 · Cited by 491
Psychological treatment of obsessive–compulsive disorder: A meta-analysis
Rosa-Alcázar, A. I., Sánchez-Meca, J., Gómez-Conesa, A., & Marín-Martínez, F. (2008) · Clinical Psychology Review · doi:10.1016/j.cpr.2008.07.001 · Cited by 444
Self-citationSnyder, H. R., Kaiser, R. H., Warren, S. L., & Heller, W. (2015) · Clinical Psychological Science · doi:10.1177/2167702614534210 · Cited by 325
“Moderator analyses were conducted only for measures with 20 or more effect sizes, given that analyses with fewer studies have inadequate power and may produce unstable estimates (Marín-Martínez & SánchezMeca, 1998; Sánchez-Meca & Marín-Martínez, 1998).”
MethodComparing meta-analytic moderator estimation techniques under realistic conditions.
Steel, P. D., & Kammeyer-Mueller, J. D. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.1.96 · Cited by 309
Psychological treatment of panic disorder with or without agoraphobia: A meta-analysis
Sánchez-Meca, J., Rosa-Alcázar, A. I., Marín-Martínez, F., & Gómez-Conesa, A. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.08.011 · Cited by 291
Self-citationA comparison of procedures to test for moderators in mixed-effects meta-regression models
Viechtbauer, W., López-López, J. A., Sánchez-Meca, J., & Marín-Martínez, F. (2015) · Psychological Methods · doi:10.1037/met0000023 · Cited by 241
Self-citationMultilevel Meta-analysis: a comparison with traditional meta-analytical procedures
Van Den Noortgate, W., & Onghena, P. (2003) · Educational and Psychological Measurement · doi:10.1177/0013164403251027 · Cited by 218
Sánchez-Meca, J., & Marín-Martínez, F. (2010) · International Encyclopedia of Education · doi:10.1016/b978-0-08-044894-7.01345-2 · Cited by 31
Self-citation
Metadata
- Authors
- Sánchez‐Meca, J., & Marín‐Martínez, F.
- Venue
- British Journal of Mathematical and Statistical Psychology
- Publisher
- Wiley
- Published
- 1998-11
- Volume
- 51
- Issue
- 2
- Pages
- 311-326
- ISSN
- 0007-1102 (ISSN-L)
- 2044-8317
- OpenAlex
- W1986452344
- Cited by
- 53
- References
- 0
- FWCI
- 2.44
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.