The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Single-Paper Meta-Analysis: Benefits for Study Summary, Theory Testing, and Replicability
Abstract
A typical behavioral research paper features multiple studies of a common phenomenon that are analyzed solely in isolation. Because the studies are of a common phenomenon, this practice is inefficient and forgoes important benefits that can be obtained only by analyzing them jointly in a single-paper meta-analysis (SPM). To facilitate SPM, we introduce meta-analytic methodology that is user-friendly, widely applicable, and specially tailored to the SPM of the set of studies that appear in a typical behavioral research paper. Our SPM methodology provides important benefits for study summary, theory testing, and replicability that we illustrate via three case studies that include papers recently published in the Journal of Consumer Research and the Journal of Marketing Research. We advocate that authors of typical behavioral research papers use it to supplement the single-study analyses that independently examine the multiple studies in the body of their papers as well as the “qualitative meta-analysis” that verbally synthesizes the studies in the general discussion of their papers. When used as such, this requires only a minor modification of current practice. We provide an easy-to-use website that implements our SPM methodology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The Handbook of Social Psychology
Gravatt, A. E., Lindzey, G., & Aronson, F. (1969) · The Family Coordinator · doi:10.2307/582242 · Cited by 12,517
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,784
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343
Statistical Methods for Research Workers
(1926) · Journal of Comparative Pathology and Therapeutics · doi:10.1016/s0368-1742(26)80018-9 · Cited by 1,283
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208
Combining results of independent studies.
Rosenthal, R. (1978) · Psychological Bulletin · doi:10.1037//0033-2909.85.1.185 · Cited by 910
Replication and Theory Development in Organizational Science: A Critical Realist Perspective
Tsang, E. W. K., & Kwan, K.-M. (1999) · The Academy of Management Review · doi:10.2307/259353 · Cited by 821
Statistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1989) · American Psychologist · doi:10.1037/0003-066x.44.10.1276 · Cited by 777
The power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“…Fabrigar and Wegener 2016; Rosenthal 1991; Stroebe and Strack 2014; Tsang and Kwan 1999), our SPM methodology estimates and accounts for heterogeneity, which has been shown to be important in a wide variety of behavioral research settings (Hedges and Pigott 2001; Klein et al. 2014; Pigott 2012).”
MethodThe Replication Recipe: What makes for a convincing replication?
Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & van 't Veer, A. (2014) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.10.005 · Cited by 705
Meta‐Analysis in Medical Research: The Handbook for the Understanding and Practice of Meta‐Analysis
(2005) · Wiley eBooks · doi:10.1002/9780470994894 · Cited by 687
Meta-Analysis in Medical Research
Sathian, B., Sreedharan, J., Ahmad, M. Z., Joy, T., Baboo, N., Dixit, S., & Devkota, S. (2009) · Cited by 652
Statistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1992) · Methodological issues & strategies in clinical research. · doi:10.1037/10109-027 · Cited by 652
The alleged crisis and the illusion of exact replication
Stroebe, W., & Strack, F. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514450 · Cited by 597
Stewart, L. A., & Tierney, J. F. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001006 · Cited by 571
Simmonds, M. C., Higginsa, J. P. T., Stewartb, L. A., Tierneyb, J. F., Clarke, M. J., & Thompson, S. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn087oa · Cited by 555
A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models
Chung, Y., Rabe-Hesketh, S., Dorie, V., Gelman, A., & Liu, J. (2013) · Psychometrika · doi:10.1007/s11336-013-9328-2 · Cited by 554
Hays, W. L. (1988) · Cited by 462
Pigott, T. D. (2012) · doi:10.1007/978-1-4614-2278-5 · Cited by 453 · cited 4×
“…Fabrigar and Wegener 2016; Rosenthal 1991; Stroebe and Strack 2014; Tsang and Kwan 1999), our SPM methodology estimates and accounts for heterogeneity, which has been shown to be important in a wide variety of behavioral research settings (Hedges and Pigott 2001; Klein et al. 2014; Pigott 2012).”
BackgroundMethodInfluentialStouffer, S. A., Suchman, E. A., Devinney, L. C., Star, S. A., & Williams, R. M. (1949) · Cited by 420
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616662243 · Cited by 398
Self-citationCooper, H., & Patall, E. A. (2009) · Psychological Methods · doi:10.1037/a0015565 · Cited by 339 · cited 1×
“…known as statistical sufficiency, to that underlying the “gold standard” metaanalytic approach—namely, an appropriately specified hierarchical (or multilevel) model fit to the individual-level observations (Cooper and Patall 2009; Haidich 2010; Simmonds et al. 2005; Stewart and Tierney 2002).”
BackgroundOn the origins of the .05 level of statistical significance.
Cowles, M., & Davis, C. (1982) · American Psychologist · doi:10.1037/0003-066x.37.5.553 · Cited by 323
Replication in Psychological Science
Lindsay, D. S. (2015) · Psychological Science · doi:10.1177/0956797615616374 · Cited by 280
Eich, E. (2014) · Psychological Science · doi:10.1177/0956797613512465 · Cited by 255
Replication Research in the Social Sciences
Madden, C. S., & Neuliep, J. W. (1992) · Journal of Marketing · doi:10.2307/1251993 · Cited by 195
Conceptualizing and evaluating the replication of research results
Fabrigar, L. R., & Wegener, D. T. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.009 · Cited by 165
Gelman, A., & Weakliem, D. L. (2009) · American Scientist · doi:10.1511/2009.79.310 · Cited by 137
If It's Useful and You Know It, Do You Eat? Preschoolers Refrain from Instrumental Food
Maimaran, M., & Fishbach, A. (2014) · Journal of Consumer Research · doi:10.1086/677224 · Cited by 127
You Cannot Step Into the Same River Twice: When Power Analyses Are Optimistic
McShane, B. B., & Böckenholt, U. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614548513 · Cited by 101
Self-citationBlinding Us to the Obvious? The Effect of Statistical Training on the Evaluation of Evidence
McShane, B. B., & Gal, D. (2015) · Management Science · doi:10.1287/mnsc.2015.2212 · Cited by 91
Self-citationPlanning sample sizes when effect sizes are uncertain: The power-calibrated effect size approach.
McShane, B. B., & Böckenholt, U. (2015) · Psychological Methods · doi:10.1037/met0000036 · Cited by 56
Self-citationSurcharges plus Unhealthy Labels Reduce Demand for Unhealthy Menu Items
Shah, A. M., Bettman, J. R., Ubel, P. A., Keller, P. A., & Edell, J. A. (2014) · Journal of Marketing Research · doi:10.1509/jmr.13.0434 · Cited by 54
A Duty to Describe: Better the Devil You Know Than the Devil You Don’t
Brown, S. D., Furrow, D. R., Hill, D. F., Gable, J. C., Porter, L. P., & Jacobs, W. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551749 · Cited by 33
Of beauty, sex, and power: Statistical challenges in estimating small eects
Gelman, A., & Weakliem, D. L. (2008) · Cited by 25
Iacobucci, D. (2005) · Journal of Consumer Research · doi:10.1086/430648 · Cited by 10
Cited by
Sarstedt, M., Hair, J. F., Pick, M., Liengaard, B. D., Radomir, L., & Ringle, C. M. (2022) · Psychology & Marketing · doi:10.1002/mar.21640 · Cited by 1,264
Mende, M., Scott, M. L., van Doorn, J., Grewal, D., & Shanks, I. (2019) · Journal of Marketing Research · doi:10.1177/0022243718822827 · Cited by 1,197
Abandon Statistical Significance
McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L. (2019) · The American Statistician · doi:10.1080/00031305.2018.1527253 · Cited by 791
Self-citationTo Err Is Human(-oid): How Do Consumers React to Robot Service Failure and Recovery?
Choi, S., Mattila, A. S., & Bolton, L. E. (2021) · Journal of Service Research · doi:10.1177/1094670520978798 · Cited by 502
Measuring Processing Fluency: One versus Five Items
Graf, L. K. M., Mayer, S., & Landwehr, J. R. (2018) · Journal of Consumer Psychology · doi:10.1002/jcpy.1021 · Cited by 318
Metadata
- Authors
- McShane, B. B., & Böckenholt, U.
- Venue
- Journal of Consumer Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-01-23
- Volume
- 43
- Issue
- 6
- Pages
- 1048-1063
- ISSN
- 0093-5301 (ISSN-L)
- 1537-5277
- OpenAlex
- W2580551816
- Cited by
- 457
- References
- 47
- FWCI
- 46.46
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.