The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
The Relative Trustworthiness of Inferential Tests of the Indirect Effect in Statistical Mediation Analysis: Does Method Really Matter?
Abstract
A content analysis of 2 years of Psychological Science articles reveals inconsistencies in how researchers make inferences about indirect effects when conducting a statistical mediation analysis. In this study, we examined the frequency with which popularly used tests disagree, whether the method an investigator uses makes a difference in the conclusion he or she will reach, and whether there is a most trustworthy test that can be recommended to balance practical and performance considerations. We found that tests agree much more frequently than they disagree, but disagreements are more common when an indirect effect exists than when it does not. We recommend the bias-corrected bootstrap confidence interval as the most trustworthy test if power is of utmost concern, although it can be slightly liberal in some circumstances. Investigators concerned about Type I errors should choose the Monte Carlo confidence interval or the distribution-of-the-product approach, which rarely disagree. The percentile bootstrap confidence interval is a good compromise test.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
Preacher, K. J., & Hayes, A. F. (2008) · Behavior Research Methods · doi:10.3758/brm.40.3.879 · Cited by 34,117
Self-citationSPSS and SAS procedures for estimating indirect effects in simple mediation models
Preacher, K. J., & Hayes, A. F. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03206553 · Cited by 17,383
Self-citationAsymptotic Confidence Intervals for Indirect Effects in Structural Equation Models
Sobel, M. E. (1982) · Sociological Methodology · doi:10.2307/270723 · Cited by 12,810
Mediation in experimental and nonexperimental studies: New procedures and recommendations.
Shrout, P. E., & Bolger, N. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.4.422 · Cited by 11,335
Addressing Moderated Mediation Hypotheses: Theory, Methods, and Prescriptions
Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007) · Multivariate Behavioral Research · doi:10.1080/00273170701341316 · Cited by 9,973
Self-citationA comparison of methods to test mediation and other intervening variable effects
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.83 · Cited by 9,406
Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3901_4 · Cited by 7,806
Introduction to Statistical Mediation Analysis
MacKinnon, D. P. (2012) · doi:10.4324/9780203809556 · Cited by 5,618
Required sample size to detect the mediated effect
Fritz, M. S., & MacKinnon, D. P. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01882.x · Cited by 4,810
Research works printed but not linked: 23
- 10.1214/aoms/1177730442 · DOI
- 10.1080/00273171.2010.498292 · DOI
- 10.1037/a0025626 · DOI · Paper page
- 10.1037/1082-989X.12.1.1 · DOI · Paper page
- 10.1080/00273171.2012.640596 · DOI
- 10.1177/0956797611411585 · DOI
- Hayes A. F., 2013, Introduction to mediation, moderation, and conditional process analysis: A regression-based approach
- 10.3758/BF03192961 · DOI · Paper page
- 10.1177/0956797611417261 · DOI
- 10.1177/0956797611423670 · DOI
- 10.1002/9780470561119.socpsy001004 · DOI · Paper page
- 10.3758/BF03192993 · DOI
- Long J. S., 2000, The American Statistician, 54, 217, 10.1080/00031305.2000.10474549 · DOI
- 10.3758/BF03193007 · DOI
- 10.1177/0956797611429132 · DOI
- Muthén L. K., 2011, Mplus user’s guide
- 10.1177/0956797611414727 · DOI
- 10.1177/0956797611435980 · DOI
- 10.1177/0956797611420882 · DOI
- 10.1177/0956797611407207 · DOI
- 10.1177/0956797611434590 · DOI
- 10.3758/s13428-011-0076-x · DOI · Paper page
- 10.1037/a0025815 · DOI
Cited by
An index and test of linear moderated mediation
Hayes, A. F. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2014.962683 · Cited by 3,979
Self-citationStatistical mediation analysis with a multicategorical independent variable
Hayes, A. F., & Preacher, K. J. (2014) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12028 · Cited by 3,403
Self-citationCommon Beliefs and Reality About PLS: Comments on Rönkkö and Evermann (2013)
Henseler, J., Dijkstra, T. K., Sarstedt, M., Ringle, C. M., Diamantopoulos, A., Straub, D. W., Ketchen, D. J., Hair, J. F., Hult, G. T. M., & Calantone, R. J. (2014) · Organizational Research Methods · doi:10.1177/1094428114526928 · Cited by 3,332
Hayes, A. F., & Rockwood, N. J. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.11.001 · Cited by 2,350
Self-citationDetermining power and sample size for simple and complex mediation models
Schoemann, A. M., Boulton, A. J., & Short, S. D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617715068 · Cited by 1,837
Gender stereotypes about intellectual ability emerge early and influence children's interests
Bian, L., Leslie, S.-J., & Cimpian, A. (2017) · Science · doi:10.1126/science.aah6524 · Cited by 1,206
PLS-SEM’s most wanted guidance
Becker, J.-M., Cheah, J.-H., Gholamzade, R., Ringle, C. M., & Sarstedt, M. (2023) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-04-2022-0474 · Cited by 1,223
Sex Differences in the Adult Human Brain: Evidence from 5216 UK Biobank Participants
Ritchie, S. J., Cox, S. R., Shen, X., Lombardo, M. V., Reus, L. M., Alloza, C., Harris, M. A., Alderson, H. L., Hunter, S., Neilson, E., et al. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy109 · Cited by 855
Haerens, L., Aelterman, N., Vansteenkiste, M., Soenens, B., & Van Petegem, S. (2015) · Psychology of Sport and Exercise · doi:10.1016/j.psychsport.2014.08.013 · Cited by 747
Guerci, M., Longoni, A., & Luzzini, D. (2016) · The International Journal of Human Resource Management · doi:10.1080/09585192.2015.1065431 · Cited by 589
Design approaches to experimental mediation
Pirlott, A. G., & MacKinnon, D. P. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.012 · Cited by 575
Power Anomalies in Testing Mediation
Kenny, D. A., & Judd, C. M. (2014) · Psychological Science · doi:10.1177/0956797613502676 · Cited by 497
Therapeutic alliance as a mediator of change: A systematic review and evaluation of research
Baier, A. L., Kline, A. C., & Feeny, N. C. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101921 · Cited by 441
Monotonicity of effect sizes: questioning kappa-squared as mediation effect size measure
Wen, Z., & Fan, X. (2015) · Psychological Methods · doi:10.1037/met0000029 · Cited by 350
Gender and leadership emergence: A meta-analysis and explanatory model
Badura, K. L., Grijalva, E., Newman, D. A., Yan, T. T., & Jeon, G. (2018) · Personnel Psychology · doi:10.1111/peps.12266 · Cited by 288
Rather, R. A., Hollebeek, L. D., & Rasoolimanesh, S. M. (2022) · Journal of Travel Research · doi:10.1177/0047287521997572 · Cited by 289
Winer, E. S., Cervone, D., Bryant, J., McKinney, C., Liu, R. T., & Nadorff, M. R. (2016) · Journal of Clinical Psychology · doi:10.1002/jclp.22298 · Cited by 260
Yan, L., Gan, Y., Ding, X., Wu, J., & Duan, H. (2021) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2020.102328 · Cited by 236
The effect of gender stereotypes on artificial intelligence recommendations
Ahn, J., Kim, J., & Sung, Y. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2021.12.007 · Cited by 207
Williams, N. J. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0693-2 · Cited by 164
Kuppens, T., Spears, R., Manstead, A. S. R., Spruyt, B., & Easterbrook, M. J. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.11.001 · Cited by 164
The “true” indirect effect won't (always) stand up: When and why reverse mediation testing fails
Lemmer, G., & Gollwitzer, M. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.05.002 · Cited by 145
Link, N. W., Ward, J. T., & Stansfield, R. (2019) · Criminology · doi:10.1111/1745-9125.12213 · Cited by 144
Peters, J. R., Smart, L. M., Eisenlohr-Moul, T. A., Geiger, P. J., Smith, G. T., & Baer, R. A. (2015) · Journal of Clinical Psychology · doi:10.1002/jclp.22189 · Cited by 112
van den Heuvel, M. I., Johannes, M. A., Henrichs, J., & Van den Bergh, B. R. H. (2015) · Early Human Development · doi:10.1016/j.earlhumdev.2014.12.003 · Cited by 105
Duan, H., Yan, L., Ding, X., Gan, Y., Kohn, N., & Wu, J. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.113396 · Cited by 97
When Parents’ Praise Inflates, Children's Self-Esteem Deflates
Brummelman, E., Nelemans, S. A., Thomaes, S., & Orobio de Castro, B. (2017) · Child Development · doi:10.1111/cdev.12936 · Cited by 97
Smith, B., Rogers, S. L., Blissett, J., & Ludlow, A. K. (2020) · Appetite · doi:10.1016/j.appet.2020.104643 · Cited by 91
Cegarra-Navarro, J.-G., & Martelo-Landroguez, S. (2020) · Journal of Intellectual Capital · doi:10.1108/jic-03-2019-0048 · Cited by 91
Habitual sleep as a contributor to racial differences in cardiometabolic risk
Curtis, D. S., Fuller-Rowell, T. E., El-Sheikh, M., Carnethon, M. R., & Ryff, C. D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618167114 · Cited by 89
BDNF genetic variants and methylation: effects on cognition in major depressive disorder
Ferrer, A., Labad, J., Salvat-Pujol, N., Barrachina, M., Costas, J., Urretavizcaya, M., de Arriba-Arnau, A., Crespo, J. M., Soriano-Mas, C., Carracedo, Á., et al. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0601-8 · Cited by 84
Panigrahy, A., Schmithorst, V. J., Wisnowski, J. L., Watson, C. G., Bellinger, D. C., Newburger, J. W., & Rivkin, M. J. (2015) · NeuroImage: Clinical · doi:10.1016/j.nicl.2015.01.013 · Cited by 86
Medlicott, E., Phillips, A., Crane, C., Hinze, V., Taylor, L., Tickell, A., Montero-Marin, J., & Kuyken, W. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18116023 · Cited by 67
Wuyts, D., Soenens, B., Vansteenkiste, M., & Van Petegem, S. (2018) · Journal of Adolescence · doi:10.1016/j.adolescence.2018.03.012 · Cited by 58
How does appreciation lead to higher job satisfaction?
Pfister, I. B., Jacobshagen, N., Kälin, W., & Semmer, N. K. (2020) · Journal of Managerial Psychology · doi:10.1108/jmp-12-2018-0555 · Cited by 57
Xu, Y., Du, J., Shahzad, F., & Li, X. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.724452 · Cited by 52
Adriaensens, S., Beyers, W., & Struyf, E. (2015) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2015.10.003 · Cited by 50
De Witte, N. A. J., Sütterlin, S., Braet, C., & Mueller, S. C. (2016) · PLOS ONE · doi:10.1371/journal.pone.0164615 · Cited by 47
Macchi, C., Favero, C., Ceresa, A., Vigna, L., Conti, D. M., Pesatori, A. C., Racagni, G., Corsini, A., Ferri, N., Sirtori, C. R., et al. (2020) · Cardiovascular Diabetology · doi:10.1186/s12933-020-01158-6 · Cited by 51
Li, Y., Gan, Z., & Zheng, B. (2025) · Information Systems Frontiers · doi:10.1007/s10796-023-10438-x · Cited by 47
Anwar, M., Khattak, M. S., Popp, J., Meyer, D. F., & Máté, D. (2020) · Technological and Economic Development of Economy · doi:10.3846/tede.2020.13404 · Cited by 46
“The other fit statistics; GFI = 0.80, AGFI = 0.80, TLI = 0.91 and NFI = 0.84 are closed to 1 or greater than 0.90 revealed suitable values (Hayes & Scharkow, 2013).”
BackgroundMethodInfluentialKangaslampi, S., & Peltonen, K. (2022) · Current Psychology · doi:10.1007/s12144-019-00478-5 · Cited by 45
“…a mechanism of change, the indirect effect should be quantified and its significance assessed by the biascorrected bootstrap confidence interval, the Monte Carlo confidence interval, the distribution of the product method or the percentile bootstrap confidence interval (Hayes and Scharkow 2013).”
BackgroundMethodInfluentialNorr, A. M., Smolenski, D. J., & Reger, G. M. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.05.009 · Cited by 44
Jia, W., Pan, L., & Neary, S. (2025) · Behavioral Sciences · doi:10.3390/bs15101348 · Cited by 45
“This study employed a bias-corrected nonparametric percentage bootstrap test, calculating 95% confidence intervals after 5000 resampling iterations to examine the total, direct, and mediating effects (Hayes & Scharkow, 2013).”
Tekala, K., Baradarani, S., Alzubi, A., & Berberoğlu, A. (2024) · Sustainability · doi:10.3390/su16135291 · Cited by 41
Knighton, J.-S., Dogan, J., Hargons, C., & Stevens-Watkins, D. (2022) · Ethnicity & Health · doi:10.1080/13557858.2020.1818695 · Cited by 38
“3 (Hayes and Scharkow 2013) in SPSS 26.0.”
InfluentialTaubner, S., Ioannou, Y., Saliba, A., Sales, C. M. D., Volkert, J., Protić, S., Adler, A., Barkauskiene, R., Conejo-Cerón, S., Di Giacomo, D., et al. (2024) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-023-02186-9 · Cited by 36
Social support, mindfulness, and job burnout of social Workers in China
Xie, X., Zhou, Y., Fang, J., & Ying, G. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.775679 · Cited by 33
Distress intolerance during smoking cessation treatment
Farris, S. G., Leyro, T. M., Allan, N. P., Øverup, C. S., Schmidt, N. B., & Zvolensky, M. J. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.08.002 · Cited by 33
Skowronski, M., Busching, R., & Krahé, B. (2021) · Sex Roles · doi:10.1007/s11199-020-01187-1 · Cited by 27
Carruthers, S. P., Rossell, S. L., Murray, G., Karantonis, J., Furlong, L. S., & Van Rheenen, T. E. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.10.107 · Cited by 24
Shapiro, M. O., Allan, N. P., Raines, A. M., & Schmidt, N. B. (2023) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-022-10002-y · Cited by 21
Montero-Marin, J., Van Gordon, W., Shonin, E., Navarro-Gil, M., Gasión, V., López-del-Hoyo, Y., Luciano, J. V., & Garcia-Campayo, J. (2020) · Mindfulness · doi:10.1007/s12671-019-01302-8 · Cited by 22
Reynolds, C. J., Makhanova, A., Ng, B. K. L., & Conway, P. (2020) · Personality and Individual Differences · doi:10.1016/j.paid.2019.109632 · Cited by 22
Kennedy, C., Deane, F. P., & Chan, A. Y. C. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.05.006 · Cited by 21
Pleiotropic Contribution of MECOM and AVPR1A to Aggression and Subcortical Brain Volumes
van Donkelaar, M. M. J., Hoogman, M., Pappa, I., Tiemeier, H., Buitelaar, J. K., Franke, B., & Bralten, J. (2018) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2018.00061 · Cited by 16
Xiong, M., Chen, J., & Ye, Y. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1111845 · Cited by 14
“Next, we used Model 4 in PROCESS macro (Hayes and Scharkow, 2013) to test the mediation effect and Model 7 to test the moderated mediation.”
MethodHu, X., Xu, X., & Chen, C. (. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2495845 · Cited by 14
Liao, Z., Chen, J., Jia, L., Chen, X., & Ding, H. (2023) · Computers & Industrial Engineering · doi:10.1016/j.cie.2023.109575 · Cited by 13
Valero-Amaro, V., Galera-Casquet, C., & Barroso-Méndez, M. J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.797621 · Cited by 13
Hou, C., Li, S., Shi, H., & Liu, Z. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1384279 · Cited by 12
Feng, Y., Lu, D., Ng, T. Y. C., Chau, P. H., Chan, L. M. L., & Kwok, J. Y. Y. (2025) · Autism · doi:10.1177/13623613251328465 · Cited by 8
Luo, B., & Yan, J. (2024) · Current Psychology · doi:10.1007/s12144-023-05234-4 · Cited by 8
Maternal parentification history impacts evaluative cognitions about self, parenting, and child
Nuttall, A. K., Ballinger, A. L., Levendosky, A. A., & Borkowski, J. G. (2021) · Infant Mental Health Journal · doi:10.1002/imhj.21912 · Cited by 7
Keser, E., & Kahya, Y. (2025) · Journal of Reproductive and Infant Psychology · doi:10.1080/02646838.2024.2335176 · Cited by 1
Paciello, M., Corbelli, G., Sportelli, C., Cicirelli, P. G., & D’Errico, F. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-08382-7 · Cited by 1
Metadata
- Authors
- Hayes, A. F., & Scharkow, M.
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2013-08-16
- Volume
- 24
- Issue
- 10
- Pages
- 1918-1927
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2129484434
- Cited by
- 2,387
- References
- 14
- FWCI
- 61.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
