The DOI matches its Crossref record; 69 of the 71 references it cites were found in a registry or an index.
How to do a meta‐analysis
Abstract
Meta‐analysis is a statistical tool for estimating the mean and variance of underlying population effects from a collection of empirical studies addressing ostensibly the same research question. Meta‐analysis has become an increasing popular and valuable tool in psychological research, and major review articles typically employ these methods. This article describes the process of conducting meta‐analysis: selecting articles, developing inclusion criteria, calculating effect sizes, conducting the actual analysis (including information on how to do the analysis on popular computer packages such as IBM SPSS and R) and estimating the effects of publication bias. Guidance is also given on how to write up a meta‐analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In terms of data extraction, all studies provided raw M ± SD scores apart from two55,56 where estimated marginal M ± SDs were given (raw data was requested from corresponding authors but could not be obtained).”
From Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials (2023) · cites this paper as [56]
In the citing paper
“Moreover, Rosenthal’s fail-safe N was calculated (to estimate how many further studies yielding zero effect would be required to make the overall ES non-significant for stress)56.”
From Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials (2023) · cites this paper as [56]
In the citing paper
“A random-effects model was selected as it was recommended that variability in effect sizes should be assumed in psychological research and that this model allowed generalisation of the findings beyond the studies included in the meta-analysis (Field and Gillett 2010).”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Field and Gillett 2010
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Discovering Statistics Using SPSS
Field, A. P., & Miles, J. N. V. (2000) · Medical Entomology and Zoology · Cited by 28,269
Self-citationOperating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META‐ANALYSIS
BARRICK, M. R., & MOUNT, M. K. (1991) · Personnel Psychology · doi:10.1111/j.1744-6570.1991.tb00688.x · Cited by 8,689
The use of ranks to avoid the assumption of normality implicit in the analysis of variance
Friedman, M. (1937) · Journal of the American Statistical Association · doi:10.1080/01621459.1937.10503522 · Cited by 7,005
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,839
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Hox, J., Moerbeek, M., & van de Schoot, R. (2002) · doi:10.4324/9781410604118 · Cited by 2,419
On the probable error of a coefficient of correlation deduced from a small sample
Fisher, R. A. (1921) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 2,089
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064 · cited 1×
“For example, when using d as an effect size measure, Orwin (1983) suggests a variation on the fail-safe N that estimates the number of unpublished studies required to bring the average effect size down to a predetermined value.”
MethodMeta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Rosenthal, R., & DiMatteo, M. R. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.59 · Cited by 1,780
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422
A comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286
Milligan, K., Astington, J. W., & Dack, L. A. (2007) · Child Development · doi:10.1111/j.1467-8624.2007.01018.x · Cited by 1,248
Some Practical Guidelines for Effective Sample Size Determination
Lenth, R. (2001) · The American Statistician · doi:10.1198/000313001317098149 · Cited by 1,200
Gender differences in temperament: A meta-analysis.
Else-Quest, N. M., Hyde, J. S., Goldsmith, H. H., & Van Hulle, C. A. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.1.33 · Cited by 1,169 · cited 1×
“On the former point, many writers have argued that real-world data in the social sciences are likely to have variable population parameters (Field, 2003b; Hunter & Schmidt, 2000, 2004; National Research Council, 1992; Osburn & Callender, 1992). There are data to support these claims: Field (2005b) calculated the stand…”
BackgroundInterpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 1×
“This bias is not trivial: significant findings are estimated to be eight times more likely to be submitted than non-significant ones (Greenwald, 1975), studies with positive findings are around seven times more likely to be published than studies with results supporting the null hypothesis (Coursol…”
ResultSterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 2×
“…than non-significant ones (Greenwald, 1975), studies with positive findings are around seven times more likely to be published than studies with results supporting the null hypothesis (Coursol & Wagner, 1986), and 97% of articles in psychology journals report significant results (Sterling, 1959).”
ResultFactors Influencing Publication of Research Results
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.1992.03480030052036 · Cited by 931
Hunter, J. E., & Schmidt, F. L. (2000) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00156 · Cited by 833 · cited 7×
“However, Hunter and Schmidt (2004) spend an entire book explaining these corrections, and so for this primer we will conduct the analysis in its simplest form.”
BackgroundMethodInfluentialStatistical Methods for Rates and Proportions.
Fielding, A., & Fleiss, J. L. (1974) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344565 · Cited by 772
Schmidt, F. L., Oh, I., & Hayes, T. L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711007x255327 · Cited by 729
Field, A. P. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.2.161 · Cited by 725
Self-citationThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“However, these tests should be used cautiously as a means to decide on how to conceptualize that data because they typically have low power to detect genuine variation in population effect sizes (Hedges & Pigott, 2001).”
BackgroundEffect-Size Indices for Dichotomized Outcomes in Meta-Analysis.
Sánchez‐Meca, J., Marín‐Martínez, F., & Moscoso, S. C. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.448 · Cited by 653
Standardized or simple effect size: what should be reported?
Baguley, T. (2009) · British Journal of Psychology · doi:10.1348/000712608x377117 · Cited by 619
The neuropsychological impact of sports-related concussion: A meta-analysis
Belanger, H. G., & Vanderploeg, R. D. (2005) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617705050411 · Cited by 547
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.2307/2282137 · Cited by 524
Bax, L., Yu, L.-M., Ikeda, N., Tsuruta, H., & Moons, K. G. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-50 · Cited by 514
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461 · cited 22×
“Opening this file in a text editor reveals the code that controls the weight functions: # Enter fixed weight function w1 ,- matrix(c(1.0,.99,.95,.90,.80,.75,.65,.60,.55,.50,.50,.50,.50,.50), ncol ¼ 1) w2 ,- matrix(c(1.0,.99,.90,.75,.60,.50,.40,.35,.30,.25,.10,.10,.10,.10), ncol ¼ 1) w3 ,- matrix(c(1.0,.99,.95,.90,.80,…”
BackgroundMethodInfluentialCartwright‐Hatton, S., Roberts, C., Chitsabesan, P., Fothergill, C., & Harrington, R. (2004) · British Journal of Clinical Psychology · doi:10.1348/0144665042388928 · Cited by 449
Confusion of confidence intervals and credibility intervals in meta-analysis.
Whitener, E. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.3.315 · Cited by 432
When effect sizes disagree: The case of r and d.
McGrath, R. E., & Meyer, G. J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.4.386 · Cited by 402
Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis
Takkouche, B., Cadarso-Suarez, C., & Spiegelman, D. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009981 · Cited by 380
Overton, R. C. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.3.354 · Cited by 365
Is the Meta-Analysis of Correlation Coefficients Accurate When Population Correlations Vary?
Field, A. P. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.444 · Cited by 329
Self-citationImplications of direct and indirect range restriction for meta-analysis methods and findings.
Hunter, J. E., Schmidt, F. L., & Le, H. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.3.594 · Cited by 315
Brewin, C. R., Kleiner, J. S., Vasterling, J. J., & Field, A. P. (2007) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.116.3.448 · Cited by 290
Self-citationHedges, L. V. (1984) · Journal of Educational Statistics · doi:10.3102/10769986009001061 · Cited by 241
The Problems in Using Fixed-Effects Models of Meta-Analysis on Real-World Data
Field, A. P. (2003) · Understanding Statistics · doi:10.1207/s15328031us0202_02 · Cited by 237
Self-citationAre parents' gender schemas related to their children's gender-related cognitions? A meta-analysis.
Tenenbaum, H. R., & Leaper, C. (2002) · Developmental Psychology · doi:10.1037/0012-1649.38.4.615 · Cited by 233
Combining Information: Statistical Issues and Opportunities for Research
Srinivasan, C. (1993) · Technometrics · doi:10.2307/1270296 · Cited by 213
Are parents' gender schemas related to their children's gender-related cognitions? A meta-analysis.
Tenenbaum, H. R., & Leaper, C. (2002) · Developmental Psychology · doi:10.1037//0012-1649.38.4.615 · Cited by 209
Meta-Analysis for Explanation: A Casebook
Spilker, B. (1993) · JAMA · doi:10.1001/jama.1993.03500230123042 · Cited by 199
Hedges, L. V. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004279 · Cited by 186
Meta-Analysis: A Comparison of Approaches
Tomasic, D. M. (2005) · Medicine & Science in Sports & Exercise · doi:10.1097/00005768-200503000-00032 · Cited by 166
Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias.
Coursol, A., & Wagner, E. E. (1986) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.17.2.136 · Cited by 165
Memory distortion in eyewitnesses: a meta-analysis of the post-identification feedback effect
Douglass, A. B., & Steblay, N. K. (2006) · Applied Cognitive Psychology · doi:10.1002/acp.1237 · Cited by 156
Validity Generalization: A Critical Review
Strube, M., & Murphy, K. R. (2004) · Academy of Management Review · doi:10.2307/20159040 · Cited by 152
Using Dissertations to Examine Potential Bias in Child and Adolescent Clinical Trials.
McLeod, B. D., & Weisz, J. R. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.2.235 · Cited by 125
Meta-Analysis of Raw Mean Differences.
Bond, C. F., Wiitala, W. L., & Richard, F. D. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.406 · Cited by 113
Comparison of two random-effects methods of meta-analysis.
Hall, S. M., & Brannick, M. T. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.2.377 · Cited by 79
Field, A. P. (2003) · Figshare · Cited by 67
Self-citationOsburn, H. G., & Callender, J. C. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.115 · Cited by 60
MetaEasy: A Meta-Analysis Add-In for MicrosoftExcel
Kontopantelis, E., & Reeves, D. (2009) · Journal of Statistical Software · doi:10.18637/jss.v030.i07 · Cited by 57
Hafdahl, A. R., & Williams, M. A. (2009) · Psychological Methods · doi:10.1037/a0014697 · Cited by 56
Improved Fisher z estimators for univariate random‐effects meta‐analysis of correlations
Hafdahl, A. R. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711008x281633 · Cited by 33
Kelley, K., Bond, R., & Abraham, C. (2001) · British Journal of Health Psychology · doi:10.1348/135910701169160 · Cited by 29
Random-effects meta-analysis of correlations: Monte Carlo evaluation of mean estimators.
Hafdahl, A. (2010) · British Journal of Mathematical & Statistical Psychology · doi:10.1348/000711009x431914 · Cited by 27
The Metric Comparability of Meta-Analytic Effect-Size Estimators From Factorial Designs.
Gillett, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.419 · Cited by 27
Self-citation
Research works printed but not linked: 12
- Clark‐Carter D., 2003, Effect size: The missing piece in the jigsaw, Psychologist, 16, 636
- Cochrane Collaboration, 2008, Review Manager (RevMan) for Windows: Version 5.0
- 10.1037/0033-2909.112.1.155 · DOI · Paper page
- DeCoster J., 1998, Microsoft Excel spreadsheets: Meta‐analysis
- Field A. P., 2005, A handbook of research methods in clinical and health psychology, 295
- Field A. P., 2009, How to do a meta‐analysis
- Fisher R. A., 1935, The design of experiments
- 10.2307/1164832 · DOI · Paper page
- Lavesque R., 2001, Syntax: Meta‐analysis
- 10.3102/10769986008002157 · DOI · Paper page
- Schwarzer G., 2005, Meta
- Wilson D. B., 2004, A spreadsheet for calculating standardized mean difference type effect sizes
Cited by
Sorted by citations
Richardson, M., Abraham, C., & Bond, R. (2012) · Psychological Bulletin · doi:10.1037/a0026838 · Cited by 3,646
Does low self-esteem predict depression and anxiety? A meta-analysis of longitudinal studies.
Sowislo, J. F., & Orth, U. (2013) · Psychological Bulletin · doi:10.1037/a0028931 · Cited by 2,123
Hayes, S. A., & Watson, S. L. (2013) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-012-1604-y · Cited by 1,847
Broadbent, J., & Poon, W. L. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.04.007 · Cited by 1,692
Much ado about grit: A meta-analytic synthesis of the grit literature
Credé, M., Tynan, M. C., & Harms, P. D. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000102 · Cited by 1,653
Metadata
- Authors
- Field, A. P., & Gillett, R.
- Affiliations
- University of Sussex (GB)
- University of Leicester (GB)
- Corresponding author
- Field, Andy P.
- Gillett, Raphael
- ORCID
- Field, Andy P.
- Venue
- British Journal of Mathematical and Statistical Psychology
- Publisher
- Wiley
- Published
- 2010-11
- Volume
- 63
- Issue
- 3
- Pages
- 665-694
- ISSN
- 0007-1102 (ISSN-L)
- 2044-8317
- OpenAlex
- W2094667953
- Cited by
- 1,162
- References
- 71
- Citations by year
- 2026: 99 · 2025: 136 · 2024: 115 · 2023: 117 · 2022: 102 · 2021: 84 · 2020: 74 · 2019: 73 · 2018: 78 · 2017: 63 · 2016: 57 · 2015: 63 · 2014: 41 · 2013: 35 · 2012: 14
- FWCI
- 18.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- British Journal of Mathematical and Statistical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Figshare (submittedVersion)
- Leicester Research Archive (University of Leicester) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- meta-analysis, effect size, publication bias, inclusion criteria, article selection, meta-analysis software
- MeSH terms
- Computer Simulation, Data Collection, Humans, Mathematical Computing, Psychology, Research, Software, Meta-Analysis as Topic, Effect Modifier, Epidemiologic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
