The meta‐analytic big bang
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
This article looks at the impact of meta‐analysis and then explores why meta‐analysis was developed at the time and by the scholars it did in the social sciences in the 1970s. For the first problem, impact, it examines the impact of meta‐analysis using citation network analysis. The impact is seen in the sciences, arts and humanities, and on such contemporaneous developments as multilevel modeling, medical statistics, qualitative methods, program evaluation, and single‐case design. Using a constrained snowball sample of citations, we highlight key articles that are either most highly cited or most central to the systematic review network. Then, the article examines why meta‐analysis came to be in the 1970s in the social sciences through the work of Gene Glass, Robert Rosenthal, and Frank Schmidt, each of whom developed similar theories of meta‐analysis at about the same time. The article ends by explaining how Simonton's chance configuration theory and Campbell's evolutionary epistemology can illuminate why meta‐analysis occurred with these scholars when it did and not in medical sciences. Copyright © 2015 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Newman (2005) · doi:10.1080/00107510500052444 · Cited by 5,749
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 2×
“In addition to the previously cited meta-analytic canon in Figure 4, central citations include Borenstein, Hedges, Higgins, and Rothstein (2011), Hedges (1981), and Light and Pillemer (1984).”
BackgroundAssessing heterogeneity in meta-analysis: Q statistic or I² index?
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.193 · Cited by 4,256
Hox, J., Moerbeek, M., & van de Schoot, R. (2010) · doi:10.4324/9780203852279 · Cited by 3,469
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
A Meta-Analysis of Cronbach's Coefficient Alpha
Peterson, R. A. (1994) · Journal of Consumer Research · doi:10.1086/209405 · Cited by 3,028
Checking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Blind variation and selective retentions in creative thought as in other knowledge processes
Campbell, D. T. (1960) · Psychological Review · doi:10.1037/h0040373 · Cited by 2,329
Deeks, J. J., Altman, D. G., & Bradburn, M. J. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch15 · Cited by 2,163
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154 · cited 1×
“…with a contingent of researchers led by the famous psychologist Hans Eysenck (1965) who concluded that psychotherapy had no more effect than a placebo and who responded to the Smith and Glass (1977) meta-analysis of psychotherapy studies by calling it an exercise in mega-silliness (Eysenck, 1978).”
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693
Systematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
The Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417
A method for assessing the quality of a randomized control trial
Chalmers, T. C., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981) · Controlled Clinical Trials · doi:10.1016/0197-2456(81)90056-8 · Cited by 1,349
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Uncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254
Interpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108
Evaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078
Popay, J., Rogers, A., & Williams, G. (1998) · Qualitative Health Research · doi:10.1177/104973239800800305 · Cited by 987
Meta-Study of Qualitative Health Research
Paterson, B., Thorne, S., Canam, C., & Jillings, C. (2001) · doi:10.4135/9781412985017 · Cited by 664
What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology.
Schmidt, F. L. (1992) · American Psychologist · doi:10.1037/0003-066x.47.10.1173 · Cited by 564
A brief history of research synthesis
Chalmers, I., Hedges, L. V., & Cooper, H. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001003 · Cited by 552
Bax, L., Yu, L.-M., Ikeda, N., Tsuruta, H., & Moons, K. G. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-50 · Cited by 514
Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews
Counsell, C. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-5-199709010-00008 · Cited by 505
The effects of psychological therapies under clinically representative conditions: a meta-analysis
Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.4.512 · Cited by 381
Self-citationSimonton, D. K. (2010) · Physics of Life Reviews · doi:10.1016/j.plrev.2010.02.002 · Cited by 261
Author Judgements about Works They Cite: Three Studies from Psychology Journals
Shadish, W. R., Tolliver, D., Gray, M., & Sen Gupta, S. K. (1995) · Social Studies of Science · doi:10.1177/030631295025003003 · Cited by 165
Self-citationRepresentative Literature Reviews Using Constrained Snowball Sampling and Citation Network Analysis
Lecy, J. D., & Beatty, K. E. (2012) · doi:10.2139/ssrn.1992601 · Cited by 133 · cited 1×
“The sampling was done using the R package CNA at a rate of 10% per level (Lecy and Beatty, 2012; Lecy and Moreda, 2013).”
MethodSelf-citationHopewell, S., Clarke, M., Lusher, A., Lefebvre, C., & Westby, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1191 · Cited by 116
Detecting and describing heterogeneity in meta-analysis
Hardy, R. J., & Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8<841::aid-sim781>3.0.co;2-d
Cited by
Meta-analysis and the science of research synthesis
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018) · Nature · doi:10.1038/nature25753 · Cited by 1,709
Research Weaving: Visualizing the Future of Research Synthesis
Nakagawa, S., Samarasinghe, G., Haddaway, N. R., Westgate, M. J., O’Dea, R. E., Noble, D. W. A., & Lagisz, M. (2019) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2018.11.007 · Cited by 261
Wood, S. G., Moxley, J. H., Tighe, E. L., & Wagner, R. K. (2018) · Journal of Learning Disabilities · doi:10.1177/0022219416688170 · Cited by 195
“Recently new methods have become available to calculate a standardized effect for single-case studies, which can be combined with between-subjects studies (Shadish & Lecy, 2015).”
MethodAssessing citation networks for dissemination and implementation research frameworks
Skolarus, T. A., Lehmann, T., Tabak, R. G., Harris, J., Lecy, J., & Sales, A. E. (2017) · Implementation Science · doi:10.1186/s13012-017-0628-2 · Cited by 163
Self-citationSchwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0_8 · Cited by 109
Metadata
- Authors
- Shadish, W. R., & Lecy, J. D.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2015-02-23
- Volume
- 6
- Issue
- 3
- Pages
- 246-264
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W1583423632
- Cited by
- 63
- References
- 33
- FWCI
- 6.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.