Overstated evidence for short-term effects of violent games on affect and behavior: A reanalysis of Anderson et al. (2010).
Open accessAbstract
Violent video games are theorized to be a significant cause of aggressive thoughts, feelings, and behaviors. Important evidence for this claim comes from a large meta-analysis by Anderson and colleagues (2010), who found effects of violent games in experimental, cross-sectional, and longitudinal research. In that meta-analysis, the authors argued that there is little publication or analytic bias in the literature, an argument supported by their use of the trim-and-fill procedure. In the present manuscript, we reexamine their meta-analysis using a wider array of techniques for detecting bias and adjusting effect sizes. Our conclusions differ from those of Anderson and colleagues in 3 salient ways. First, we detect substantial publication bias in experimental research on the effects of violent games on aggressive affect and aggressive behavior. Second, after adjustment for bias, the effects of violent games on aggressive behavior in experimental research are estimated as being very small, and estimates of effects on aggressive affect are much reduced. In contrast, the cross-sectional literature finds correlations that appear largely unbiased. Third, experiments meeting the original authors' criteria for methodological quality do not yield larger adjusted effects than other experiments, but instead yield larger indications of bias, indicating that perhaps they were selected for significance. We outline future directions for stronger experimental research. The results indicate th…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies originally deemed to be following best practices showed particularly strong evidence of publication bias.
“this analysis also suggested that studies originally deemed to be following best practices showed particularly strong evidence of publication bias.”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
Upon adjusting for publication bias, the observed aggregate effect sizes relating gaming to aggression were smaller than those originally presented.
“Upon adjusting for publication bias, the observed aggregate effect sizes relating gaming to aggression were smaller than those originally presented.”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
The existing literature suggests that the effect linking violent video game play to aggressive behaviour in cross-sectional research (n = 29 113) is approximately r = 0.21 (95% CI = 0.20-0.22).
“The existing literature suggests that the effect linking violent video game play to aggressive behaviour in cross-sectional research (n = 29 113) is approximately r = 0.21 (95% CI = 0.20–0.22).”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
The observed effect r = 0.01 did not overlap with and was clearly inferior to the meta-analytic estimate r = 0.21 (95% CI = 0.20-0.22).
“Given this effect, r = 0.01 (95% CI = −0.08 to 0.10), did not overlap with, and was clearly inferior to, r = 0.21 (95% CI = 0.20–0.22)”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
Hilgard et al. re-analysed widely cited meta-analytic data and detected publication bias not uncovered in the original reporting.
“Their analysis detected the presence of publication bias not uncovered in the original reporting of the data.”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
We prespecified how we would interpret effects larger, smaller or falling in the same range as estimates from meta-analysis.
“because we prespecified how we would interpret effects larger, smaller or falling in the same range as estimates from meta-analysis [32]”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
James, W. (1891) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,799 · see citing works
Conducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · see citing works
The unbearable automaticity of being
Bargh, J. A., & Chartrand, T. L. (1999) · American Psychologist · doi:10.1037/0003-066x.54.7.462 · Cited by 3,699 · see citing works
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · see citing works
Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein, H. R., & Saleem, M. (2010) · Psychological Bulletin · doi:10.1037/a0018251 · Cited by 2,048 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life.
Anderson, C. A., & Dill, K. E. (2000) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.78.4.772 · Cited by 1,778 · see citing works
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766 · see citing works
Meta-analysis of individual participant data: rationale, conduct, and reporting
Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010) · BMJ · doi:10.1136/bmj.c221 · Cited by 1,701 · see citing works
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · see citing works
A Multilab Preregistered Replication of the Ego-Depletion Effect
Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H. J. E. M., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brand, M., Brewer, G., Bruyneel, S. D., et al. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616652873 · Cited by 1,097 · see citing works
Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified
Higgins, J. P. T. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn204 · Cited by 995 · see citing works
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916 · see citing works
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809 · see citing works
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751 · see citing works
<i>p</i> -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717 · see citing works
(2009) · Cited by 670 · see citing works
Video Games Do Affect Social Outcomes
Greitemeyer, T., & Mügge, D. O. (2014) · Personality and Social Psychology Bulletin · doi:10.1177/0146167213520459 · Cited by 636 · see citing works
Violent Video Games and Hostile Expectations: A Test of the General Aggression Model
Bushman, B. J., & Anderson, C. A. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/014616702237649 · Cited by 615 · see citing works
Ferguson, C. J. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615592234 · Cited by 524 · see citing works
Carter, E. C., & McCullough, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00823 · Cited by 381 · see citing works
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373 · see citing works
A general linear model for estimating effect size in the presence of publication bias
Vevea, J. L., & Hedges, L. (1995) · Psychometrika · doi:10.1007/bf02294384 · Cited by 367 · see citing works
Evidence for publication bias in video game violence effects literature: A meta-analytic review
Ferguson, C. J. (2007) · Aggression and Violent Behavior · doi:10.1016/j.avb.2007.01.001 · Cited by 364 · see citing works
The Public Health Risks of Media Violence: A Meta-Analytic Review
Ferguson, C. J., & Kilburn, J. C. (2009) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2008.11.033 · Cited by 330 · see citing works
Ferguson, C. J., & Kilburn, J. (2010) · Psychological Bulletin · doi:10.1037/a0018566 · Cited by 311 · see citing works
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265 · see citing works
Competence-impeding electronic games and players' aggressive feelings, thoughts, and behaviors
Przybylski, A. K., Deci, E. L., Rigby, C. S., & Ryan, R. M. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0034820 · Cited by 263 · see citing works
The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles
O’Boyle, E. H., Banks, G. C., & Gonzalez-Mulé, E. (2017) · Journal of Management · doi:10.1177/0149206314527133 · Cited by 246 · see citing works
Priming Intelligent Behavior: An Elusive Phenomenon
Shanks, D. R., Newell, B. R., Lee, E. H., Balakrishnan, D., Ekelund, L., Cenac, Z., Kavvadia, F., & Moore, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0056515 · Cited by 240 · see citing works
Elson, M., & Ferguson, C. J. (2014) · European Psychologist · doi:10.1027/1016-9040/a000147 · Cited by 237 · see citing works
On the reproducibility of meta-analyses: six practical recommendations
Lakens, D., Hilgard, J., & Staaks, J. P. C. (2016) · BMC Psychology · doi:10.1186/s40359-016-0126-3 · Cited by 231 · see citing works
Self-citationvan Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219 · see citing works
Ballard, M. E., & Wiest, J. (1996) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1996.tb02740.x · Cited by 196 · see citing works
Willoughby, T., Adachi, P. J. C., & Good, M. (2011) · Developmental Psychology · doi:10.1037/a0026046 · Cited by 178 · see citing works
Elson, M., Mohseni, M. R., Breuer, J., Scharkow, M., & Quandt, T. (2014) · Psychological Assessment · doi:10.1037/a0035569 · Cited by 159 · see citing works
The effect of horizontal eye movements on free recall: A preregistered adversarial collaboration.
Matzke, D., Nieuwenhuis, S., van Rijn, H., Slagter, H. A., van der Molen, M. W., & Wagenmakers, E. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000038 · Cited by 114 · see citing works
Huesmann, L. R. (2010) · Psychological Bulletin · doi:10.1037/a0018567 · Cited by 107 · see citing works
Does Cleanliness Influence Moral Judgments?
Johnson, D. J., Cheung, F., & Donnellan, M. B. (2014) · Social Psychology · doi:10.1027/1864-9335/a000186 · Cited by 105 · see citing works
Bushman, B. J., Rothstein, H. R., & Anderson, C. A. (2010) · Psychological Bulletin · doi:10.1037/a0018718 · Cited by 104 · see citing works
Graybill, D. F., Kirsch, J. R., & Esselman, E. D. (1985) · Child study journal · Cited by 99 · see citing works
Violent video games and physical aggression: Evidence for a selection effect among adolescents.
Breuer, J., Vogelgesang, J., Quandt, T., & Festl, R. (2015) · Psychology of Popular Media Culture · doi:10.1037/ppm0000035 · Cited by 98 · see citing works
An Empirical Investigation of Partial Effect Sizes in Meta-Analysis of Correlational Data
Aloe, A. M. (2013) · The Journal of General Psychology · doi:10.1080/00221309.2013.853021 · Cited by 92 · see citing works
Angry Birds, Angry Children, and Angry Meta-Analysts
Furuya‐Kanamori, L., & Doi, S. A. R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616635599 · Cited by 85 · see citing works
Boxer, P., Groves, C. L., & Docherty, M. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615592239 · Cited by 68 · see citing works
DeWall, C. N., & Bushman, B. J. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.05.003 · Cited by 61 · see citing works
Etchells, P. J., Gage, S. H., Rutherford, A. D., & Munafo, M. R. (2016) · PLoS ONE · doi:10.1371/journal.pone.0147732 · Cited by 61 · see citing works
Kneer, J., Elson, M., & Knapp, F. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.07.034 · Cited by 59 · see citing works
A Bayesian approach to mitigation of publication bias
Guan, M., & Vandekerckhove, J. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0868-6 · Cited by 59 · see citing works
Extreme between-study homogeneity in meta-analyses could offer useful insights
Ioannidis, J. P. A., Trikalinos, T. A., & Zintzaras, E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.013 · Cited by 58 · see citing works
Panee, C. D., & Ballard, M. E. (2002) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2002.tb02751.x · Cited by 52 · see citing works
Engelhardt, C. R., Mazurek, M. O., Hilgard, J., Rouder, J. N., & Bartholow, B. D. (2015) · Psychological Science · doi:10.1177/0956797615583038 · Cited by 49 · see citing works
Self-citationElson, M., Breuer, J., LOOY, J. V., Kneer, J., & Quandt, T. (2013) · Psychology of Popular Media Culture · doi:10.1037/ppm0000010 · Cited by 47 · see citing works
Engelhardt, C. R., Bartholow, B. D., & Saults, J. S. (2011) · Aggressive Behavior · doi:10.1002/ab.20411 · Cited by 43 · see citing works
Self-citationTear, M. J., & Nielsen, M. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.09.002 · Cited by 41 · see citing works
Sigurðsson, J. F., Gudjonsson, G. H., Bragason, A. V., Kristjansdottir, E., & Sigfúsdóttir, I. D. (2006) · Personality and Individual Differences · doi:10.1016/j.paid.2006.02.006 · Cited by 39 · see citing works
Digital games in laboratory experiments: Controlling a complex stimulus through modding.
Elson, M., & Quandt, T. (2014) · Psychology of Popular Media Culture · doi:10.1037/ppm0000033 · Cited by 35 · see citing works
The consistency test does not–and cannot–deliver what is advertised: A comment on Francis (2013)
Morey, R. D. (2013) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2013.03.004 · Cited by 26 · see citing works
Acute exposure to difficult (but not violent) video games dysregulates cognitive control
Engelhardt, C. R., Hilgard, J., & Bartholow, B. D. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.11.089 · Cited by 20 · see citing works
Self-citationClose replication attempts of the heat priming-hostile perception effect
McCarthy, R. J. (2014) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2014.04.014 · Cited by 13 · see citing works
Violent Media Effects on Aggression: A Commentary from a Cross‐Cultural Perspective
Krahé, B. (2016) · Analyses of Social Issues and Public Policy · doi:10.1111/asap.12107 · Cited by 5 · see citing works
Engelhardt, C. R., Mazurek, M. O., Hilgard, J., Rouder, J. N., & Bartholow, B. D. (2014) · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588 · see citing works
Self-citationMedia Use Is Linked to Lower Psychological Well-Being: Evidence from Three Datasets
Twenge, J. M., & Campbell, W. K. (2019) · Psychiatric Quarterly · doi:10.1007/s11126-019-09630-7 · Cited by 429 · see citing works
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354 · see citing works
“…could be explained away by publication bias; other researchers suggested that publication bias might largely explain these results (Ferguson and Kilburn, 2010; Hilgard et al., 2017), whereas the original authors argued that the results were in fact robust to publication bias (Kepes et al., 2017).)”
ResultWhen and How Video Games Can Be Good: A Review of the Positive Effects of Video Games on Well-Being
Halbrook, Y. J., O’Donnell, A. T., & Msetfi, R. M. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619863807 · Cited by 263 · see citing works
Beyond Screen Time: Identity Development in the Digital Age
Granic, I., Morita, H., & Scholten, H. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1820214 · Cited by 237 · see citing works
Przybylski, A. K., & Weinstein, N. (2019) · Royal Society Open Science · doi:10.1098/rsos.171474 · Cited by 228 · see citing works
“The estimate we are using for the power analysis for our proposed study is based on the naïve effect-size estimates in Table 1 (3) for cross sectional studies relating violent game play to self-report measures of aggressive (k = 37, n = 29,113) which yields an estimated an effect size of r = .”
MethodVideo games, emotion, and emotion regulation: expanding the scope
Hemenover, S. H., & Bowman, N. D. (2018) · Annals of the International Communication Association · doi:10.1080/23808985.2018.1442239 · Cited by 159 · see citing works
“…of the gaming-aggression association is beyond the scope of the current manuscript, meta-analyses (Anderson et al., 2010; Greitemeyer & Mügge, 2014) found support for a moderate association, although these findings have been challenged (cf. Ferguson, 2007; Hilgard, Engelhardt, & Rouder, 2017).”
BackgroundResultRobust Bayesian meta-analysis: Addressing publication bias with model-averaging.
Maier, M., Bartos̆, F., & Wagenmakers, E. (2022) · Psychological Methods · doi:10.1037/met0000405 · Cited by 144 · see citing works
Drummond, A., Sauer, J. D., & Ferguson, C. J. (2020) · Royal Society Open Science · doi:10.1098/rsos.200373 · Cited by 106 · see citing works
Aggressive Video Games are Not a Risk Factor for Future Aggression in Youth: A Longitudinal Study
Ferguson, C. J., & Wang, J. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01069-0 · Cited by 102 · see citing works
“However, reanalysis of Anderson et al. (2010) has suggested that publication bias inflated outcomes, particularly for experimental studies (Hilgard et al. 2017).”
BackgroundFinding Common Ground in Meta-Analysis “Wars” on Violent Video Games
Mathur, M. B., & VanderWeele, T. J. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619850104 · Cited by 96 · see citing works
Ferguson, C. J., Copenhaver, A., & Markey, P. M. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620927666 · Cited by 95 · see citing works
“Since the APA task-force technical report was released, several other meta-analyses have been released, each coming to different conclusions (e.g., Ferguson, 2015a; Furuya-Kanamori & Doi, 2016; Hilgard et al., 2017; Prescott et al., 2018).”
BackgroundMethodPerry, R., Drachen, A., Kearney, A., Kriglstein, S., Nacke, L. E., Sifa, R., Wallner, G., & Johnson, D. M. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.10.032 · Cited by 93 · see citing works
A Meta-Analysis on the Longitudinal, Age-Dependent Effects of Violent Video Games on Aggression
Burkhardt, J., & Lenhard, W. (2021) · Media Psychology · doi:10.1080/15213269.2021.1980729 · Cited by 89 · see citing works
Przybylski, A. K., & Weinstein, N. (2019) · Clinical Psychological Science · doi:10.1177/2167702619859341 · Cited by 74 · see citing works
The dark and bright side of video game consumption: Effects of violent and prosocial video games
Greitemeyer, T. (2022) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2022.101326 · Cited by 74 · see citing works
Null Effects of Game Violence, Game Difficulty, and 2D:4D Digit Ratio on Aggressive Behavior
Hilgard, J., Engelhardt, C. R., Rouder, J. N., SEGERT, I., & Bartholow, B. D. (2019) · Psychological Science · doi:10.1177/0956797619829688 · Cited by 55 · see citing works
Self-citationNeglect of publication bias compromises meta-analyses of educational research
Ropovik, I., Adamkovic, M., & Greger, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252415 · Cited by 52 · see citing works
Validity Threats in Quantitative Data Collection With Games: A Narrative Survey
Gundry, D., & Deterding, S. (2018) · Simulation & Gaming · doi:10.1177/1046878118805515 · Cited by 46 · see citing works
“Another approach is to adapt a single game to provide treatment and control conditions, a so-called modified game paradigm (Hilgard, Engelhardt, & Rouder, 2017).”
Riet, J. V. ‘., Meeuwes, A. C., van der Voorden, L., & Jansz, J. (2018) · Basic and Applied Social Psychology · doi:10.1080/01973533.2018.1459301 · Cited by 45 · see citing works
Kepes, S., Bushman, B. J., & Anderson, C. A. (2017) · Psychological Bulletin · doi:10.1037/bul0000112 · Cited by 43 · see citing works
Hartanto, A., Lua, V. Y. Q., Quek, F. Y. X., Yong, J. C., & Ng, M. H. S. (2021) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2021.100135 · Cited by 45 · see citing works
Drummond, A., & Sauer, J. D. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103704 · Cited by 42 · see citing works
Sex Doll Ownership: An Agenda for Research
Harper, C. A., & Lievesley, R. (2020) · Current Psychiatry Reports · doi:10.1007/s11920-020-01177-w · Cited by 42 · see citing works
Aggressive video games research emerges from its replication crisis (Sort of)
Ferguson, C. J. (2020) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2020.01.002 · Cited by 41 · see citing works
Video games, frustration, violence, and virtual reality: Two studies
Ferguson, C. J., Gryshyna, A., Kim, J. S., Knowles, E., Nadeem, Z., Cardozo, I., Esser, C., Trebbi, V., & Willis, E. (2021) · British Journal of Social Psychology · doi:10.1111/bjso.12471 · Cited by 39 · see citing works
“…et al., 2010);however, a reanalysis of this meta found that, for experimental studies, effects were likely inflated due to publication bias, with the true effect size difficult to infer from this group of studies (Hilgard, Engelhardt, Bartholow, & Rouder, 2017; Hilgard, Engelhardt, & Rouder, 2017).”
BackgroundDrummond, A., Sauer, J. D., Ferguson, C. J., Cannon, P. R., & Hall, L. C. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104119 · Cited by 37 · see citing works
Schlegel, L., & Kowert, R. (2024) · Routledge eBooks · doi:10.4324/9781003388371 · Cited by 35 · see citing works
Teng, Z., Nie, Q., Liu, Y., & Guo, C. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.10.014 · Cited by 29 · see citing works
Behavioural realism and the activation of aggressive concepts in violent video games
Zendle, D. I., Kudenko⋆, D., & Cairns, P. A. (2017) · Entertainment Computing · doi:10.1016/j.entcom.2017.10.003 · Cited by 29 · see citing works
Emotion perception in habitual players of action video games.
Pichon, S., Bediou, B., Antico, L., Jack, R. E., Garrod, O., Sims, C. R., Green, C. S., Schyns, P. G., & Bavelier, D. (2020) · Emotion · doi:10.1037/emo0000740 · Cited by 29 · see citing works
Zioga, T., Ferentinos, A., Konsolaki, E., Nega, C., & Kourtesis, P. (2024) · Behavioral Sciences · doi:10.3390/bs14100874 · Cited by 28 · see citing works
“However, this perspective has been challenged by studies arguing that the deleterious effects of violent video games are overstated, and result from less robust methodologies [28–33].”
BackgroundThe Influence of Empathy and Morality of Violent Video Game Characters on Gamers’ Aggression
Gao, X., Weng, L., Zhou, Y., & Yu, H. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01863 · Cited by 28 · see citing works
“Further, there is debate about the relationship between violent video games and empathy and aggression (Gao et al., 2017; Hilgard et al., 2017; Kepes et al., 2017; Szycik et al., 2017).”
BackgroundLópez-Fernández, F. J., Mezquita, L., Etkin, P., Griffiths, M. D., Ortet, G., & Ibáñez, M. I. (2020) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2020.0030 · Cited by 28 · see citing works
Ruiz-Fernández, A., Junco-Guerrero, M., & Cantón-Cortés, D. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18062845 · Cited by 26 · see citing works
“Previous research has suggested that this disparity in results may be due, in addition to the biases of the researchers, which might resemble past moral panics with attitudes that reflect generational conflicts [28], to publication biases [29].”
BackgroundHo, M., Huyền, N. T. T., Nguyen, M., La, V., & Vuong, Q. (2022) · AMBIO · doi:10.1007/s13280-022-01742-y · Cited by 26 · see citing works
“However, this analysis was argued to have substantial publication bias in experimental research, and the adjustment of which resulted in a reduction of aggression effect (Hilgard et al. 2017).”
BackgroundA Manifesto for More Productive Psychological Games Research
Ballou, N. (2023) · Games Research and Practice · doi:10.1145/3582929 · Cited by 25 · see citing works
“These problems are just as relevant to research on games, with studies finding that statistical reporting is often poor [142] and that many topics are characterized by substantial publication bias [54, 56].”
BackgroundThe Relation between Electronic Game Play and Executive Function among Preschoolers
Yang, X., Wang, Z., Qiu, X., & Zhu, L. (2020) · Journal of Child and Family Studies · doi:10.1007/s10826-020-01754-w · Cited by 23 · see citing works
Who plays violent video games? An exploratory analysis of predictors of playing violent games
DeCamp, W. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.06.027 · Cited by 22 · see citing works
Has Stereotype Threat Dissipated Over Time? A Cross-Temporal Meta-AnalysisPreprint
Lewis, N. A., & Michalak, N. M. (2019) · doi:10.31234/osf.io/w4ta2 · Cited by 20 · see citing works
Virtual tree, real impact: how simulated worlds associate with the perception of limited resources
Ho, M., Huyền, N. T. T., Nguyen, M., La, V., & Vuong, Q. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-022-01225-1 · Cited by 21 · see citing works
“…games was found in numerous experimental, cross-sectional, and longitudinal studies (Anderson et al. 2010), its result is subjected to significant publication bias presented in empirical research, and the aggression effect decreased substantially upon the removal of this bias (Hilgard et al. 2017).”
BackgroundGranic, I., Morita, H., & Scholten, H. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1820225 · Cited by 23 · see citing works
Does sexualization in video games cause harm in players? A meta-analytic examination
Ferguson, C. J., Sauer, J. D., Drummond, A., Kneer, J., & Lowe-Calverley, E. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107341 · Cited by 19 · see citing works
“Our results appear to fit with increased scrutiny in other areas of media research such as in video game violence (Hilgard et al., 2017), thin ideal media (Want & Saiphoo, 2017; Whyte et al., 2016) or movie smoking and teen smoking (Ferguson et al., 2020).”
BackgroundResultA new variation of modern prejudice: young Korean men's anti-feminism and male-victim ideology
Jung, H. W. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1230577 · Cited by 20 · see citing works
Violent Video Games, Sexist Video Games, and the Law: Why Can't We Find Effects?
Ferguson, C. J. (2018) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-101317-031036 · Cited by 19 · see citing works
Tunç, D. U., & Tunç, M. N. (2020) · doi:10.31234/osf.io/pdm7y · Cited by 20 · see citing works
“…so much resource has been invested in the empirical investigation of these almost-too-relevant problems, nothing much is accomplished in terms of arriving at clear, definitive answers (see Barrett et al., 2019; Ellemers et al., 2020; Hilgard et al., 2017; Lin et al., 2020; Wicherts et al., 2010).”
BackgroundAssociations Between Different Aspects of Video Game Play Behavior and Adolescent Adjustment
Verheijen, G. P., Burk, W. J., Stoltz, S. E. M. J., van den Berg, Y. H. M., & Cillessen, A. H. N. (2019) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000253 · Cited by 18 · see citing works
Action Video-Game Training and Its Effects on Perception and Attentional Control
Green, C. S., Gorman, T. E., & Bavelier, D. (2016) · Springer eBooks · doi:10.1007/978-3-319-42662-4_10 · Cited by 19 · see citing works
Drummond, A., Sauer, J. D., & Garea, S. S. (2018) · Springer eBooks · doi:10.1007/978-3-319-95495-0_3 · Cited by 17 · see citing works
“Assuming that the effects of VVGs are large enough to consider an important predictor of aggression (i.e., the lowest estimates by Hilgard et al., 2017 are incorrect and the true effect exceeds r = .1), it remains somewhat unclear why these effects occur.”
InfluentialVideo Game Violence and Interactivity: Effect or Equivalence?
Weber, R., Behr, K. M., Fisher, J. T., Lonergan, C., & Quebral, C. (2020) · Journal of Communication · doi:10.1093/joc/jqz048 · Cited by 18 · see citing works
Lengersdorff, L. L., Wagner, I. C., Mittmann, G., Sastre-Yagüe, D., Lüttig, A., Olsson, A., Petrovic, P., & Lamm, C. (2023) · eLife · doi:10.7554/elife.84951 · Cited by 10 · see citing works
“Other researchers contest these results, claiming that results are a product of selective reporting and biased analyses ( Ferguson and Kilburn, 2010 ; Hilgard et al., 2017b ).”
ResultModelling publication bias and p‐hacking
Moss, J., & de Bin, R. (2019) · Biometrics · doi:10.1111/biom.13560 · Cited by 20 · see citing works
Zhao, H., Zhou, J., Xu, X., Gong, X., Zheng, J., & Zhou, J. (2020) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2019.0762 · Cited by 16 · see citing works
“Six mother-reported items from the State-Trait Anger Expression Inventory-2 44 were used to assess their child’s anger.”
Violent media use and aggression: Two longitudinal network studies
Delhove, M., & Greitemeyer, T. (2021) · The Journal of Social Psychology · doi:10.1080/00224545.2021.1896465 · Cited by 19 · see citing works
Międzobrodzka, E., Buczny, J., Konijn, E. A., & Krabbendam, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.651759 · Cited by 17 · see citing works
Prosocial Video Game Content, Empathy and Cognitive Ability in a Large Sample of Youth
García, S., Ferguson, C. J., & Wang, C. K. J. (2021) · Journal of Youth and Adolescence · doi:10.1007/s10964-021-01512-1 · Cited by 16 · see citing works
“, 2010), although reanalysis of this metaanalysis suggested publication bias may have elevated effect sizes (Hilgard et al., 2017) and other meta-analyses”
BackgroundViolent Video Games Do Contribute to Aggression
Scharrer, E., Kamau, G., Warren, S., & Zhang, C. (2018) · Springer eBooks · doi:10.1007/978-3-319-95495-0_2 · Cited by 14 · see citing works
Pan, W., Gao, X., Shi, S., Liu, F., & Li, C. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02219 · Cited by 15 · see citing works
“…by increasing number of literatures claiming that the severe effect of violent video games is overstated (Ferguson et al., 2008, 2017a,b; Decety et al., 2009; Ferguson, 2010; Collins and Freeman, 2013; DeCamp and Ferguson, 2017; Gao et al., 2017; Hilgard et al., 2017; Szycik et al., 2017a,b).”
BackgroundFerguson, C. J., & Colwell, J. (2020) · Criminal Behaviour and Mental Health · doi:10.1002/cbm.2138 · Cited by 14 · see citing works
Emotion Recognition and Inhibitory Control in Adolescent Players of Violent Video Games
Międzobrodzka, E., Konijn, E. A., & A, L. K. M. (2021) · Journal of Research on Adolescence · doi:10.1111/jora.12704 · Cited by 14 · see citing works
Personality and frustration predict aggression and anger following violent media
Devilly, G. J., O’Donohue, R. P., & Brown, K. M. (2021) · Psychology Crime and Law · doi:10.1080/1068316x.2021.1999949 · Cited by 14 · see citing works
“Together with other factors, this lead Hilgard et al. (2017) to conclude that publication bias severely overestimated the relationships between violent video gaming and negative emotional consequences.”
BackgroundAdolescent Aggression: A Narrative Review on the Potential Impact of Violent Video Games
Borrego-Ruiz, A., & Borrego, J. J. (2025) · Psychology International · doi:10.3390/psycholint7010012 · Cited by 13 · see citing works
“…remain about the strength of the positive relationship between VVGs and aggressive behavior among adolescents (Cunningham et al., 2016; Ewell et al., 2025; Ferguson, 2020; Ferguson et al., 2020, 2022; Goodson et al., 2021; Hilgard et al., 2017; Lengersdorff et al., 2023; Ritchie et al., 2024).”
Videogames and guns in adolescents: T ests of a bipartite theory
Turel, O. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106355 · Cited by 13 · see citing works
Hawk, C. E., & Ridge, R. D. (2021) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000291 · Cited by 13 · see citing works
“Pro Gamers” & Cyberbullying: Workplace bullying & sexual harassment in professional video gaming
Trudgett-Klose, L. H., & McLinton, S. S. (2024) · Entertainment Computing · doi:10.1016/j.entcom.2024.100702 · Cited by 15 · see citing works
Effects of violent media content: Evidence from the rise of the UFC.
Lindo, J., Swensen, I. D., & Waddell, G. (2022) · Journal of Health Economics · doi:10.1016/j.jhealeco.2022.102623 · Cited by 10 · see citing works
“As a whole, though they have likely overstated these effects (Hilgard et al., 2017), these laboratory studies seem to provide compelling evidence that viewing violent media can increase aggressive anti-social tendencies.”
ResultFalsificationist Treatment of Auxiliary Hypotheses in Social and Behavioral Sciences
Tunç, D. U., & Tunç, M. N. (2023) · Meta-Psychology · doi:10.15626/mp.2021.2756 · Cited by 32 · see citing works
The (Not So) Evil Within? Agency in Video Game Choice and the Impact of Violent Content
Ferguson, C. J., Colon-Motas, K., Esser, C., Lanie, C., Purvis, S., & Williams, M. G. (2016) · Simulation & Gaming · doi:10.1177/1046878116683521 · Cited by 11 · see citing works
Sensitivity analysis for publication bias in meta-analysesPreprint
Mathur, M. B., & VanderWeele, T. J. (2019) · doi:10.31219/osf.io/s9dp6 · Cited by 11 · see citing works
Jin, M., Onie, S., Curby, K. M., & Most, S. B. (2018) · Visual Cognition · doi:10.1080/13506285.2018.1553223 · Cited by 10 · see citing works
“Thus, although there is debate about how robust the links may be between violent video game exposure and outward, complex behaviour (Hilgard et al., 2017), the current study raises the possibility of observable links between frequent violent video game use and early stages of information…”
BackgroundRetracted: Effects of cartoon violence on aggressive thoughts and aggressive behaviors
Zhang, Q., Cao, Y., Gao, J., Yang, X., Rost, D. H., Cheng, G., Teng, Z., & Espelage, D. L. (2019) · Aggressive Behavior · doi:10.1002/ab.21836 · Cited by 10 · see citing works
RetractedDelhove, M., & Greitemeyer, T. (2019) · International Review of Social Psychology · doi:10.5334/irsp.242 · Cited by 10 · see citing works
How peer influence mediates the effects of video games playing on adolescents’ aggressive behavior
Sun, Y., & Sun, M. (2021) · Children and Youth Services Review · doi:10.1016/j.childyouth.2021.106225 · Cited by 10 · see citing works
Video Games, Violence Justification and Child-to-Parent Violence
Junco-Guerrero, M., Ruiz-Fernández, A., & Cantón-Cortés, D. (2024) · Journal of Child and Family Studies · doi:10.1007/s10826-024-02864-5 · Cited by 9 · see citing works
“An example of the need for caution in such research is the study by Hilgard et al. (2017), who re-analysed meta-analytical data on this issue, bringing together results on GAM research.”
Hodge, S., Taylor, J., & McAlaney, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01313 · Cited by 10 · see citing works
Stress and Violence in Video Games: Their Influence on Aggression
Barrington, G., & Ferguson, C. J. (2022) · Trends in Psychology · doi:10.1007/s43076-022-00141-2 · Cited by 9 · see citing works
“However, a reanalysis of this data concluded that effects, particularly for experimental studies, were largely driven by publication bias (Hilgard et al., 2017).”
Mothers, fathers and educators’ beliefs about play in Chilean preschool children
Aldoney, D., Coo, S., Mira, A., & Valdivia, J. (2022) · International Journal of Play · doi:10.1080/21594937.2022.2069346 · Cited by 9 · see citing works
“The majority of the existing studies explore the effect of electronic games on aggression (Hilgard et al., 2017), attention (Zivan et al., 2019), and executive function (Yang et al., 2020) and have yielded mixed results; thus suggesting that we still have a very limited understanding of the…”
BackgroundThe Basic Needs in Games Model of Video Game Play and Mental Health
Ballou, N., & Deterding, S. (2024) · Interacting with Computers · doi:10.1093/iwc/iwae042 · Cited by 14 · see citing works
Wagener, G. L., Schulz, A., & Melzer, A. (2023) · Physiology & Behavior · doi:10.1016/j.physbeh.2023.114421 · Cited by 8 · see citing works
Devasia, N., Kenderova, G., Newman, M., Kientz, J. A., & Lee, J. H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3748633 · Cited by 7 · see citing works
“However, this is a topic of ongoing debate, e.g., [42, 46, 47], and the literature faces continuing concerns surrounding publication bias [45, 70].”
ADOLESCENT VIDEO GAME PLAYING AND FIGHTING OVER THE LONG‐TERM
Ward, M. R. (2019) · Contemporary Economic Policy · doi:10.1111/coep.12451 · Cited by 7 · see citing works
Parental influence on youth violent video game use
DeCamp, W. (2019) · Social Science Research · doi:10.1016/j.ssresearch.2019.04.013 · Cited by 7 · see citing works
It Does Matter If You Win or Lose, and How You Play the (Video) Game
Hemovich, V. (2020) · Games and Culture · doi:10.1177/1555412020913760 · Cited by 7 · see citing works
Video game playing frequency, social cognition, and social behavior in childhood.
Lavertu, G., Seni, A. G., Vera-Estay, E., Tran, M., West, G. L., & Beauchamp, M. H. (2022) · Technology Mind and Behavior · doi:10.1037/tmb0000075 · Cited by 6 · see citing works
Teaching Students About Violent Media Effects
Bushman, B. J. (2018) · Teaching of Psychology · doi:10.1177/0098628318762936 · Cited by 6 · see citing works
“Some scholars have questioned how robust the meta-analytic results are for violent video game effects, due to possible publication bias (see Hilgard, Engelhardt & Rouder, 2017; but also see response to Hilgard et al., 2017, by Kepes, Bushman, & Anderson, 2017 for a response).”
BackgroundAction Video Game Training and Its Effects on Perception and Attentional Control
Bediou, B., Bavelier, D., & Green, C. S. (2020) · Springer eBooks · doi:10.1007/978-3-030-39292-5_15 · Cited by 6 · see citing works
Comment on Yoon and Vargas (2014): An Implausibly Large Effect From Implausibly Invariant Data
Hilgard, J. (2019) · Psychological Science · doi:10.1177/0956797618815434 · Cited by 6 · see citing works
Self-citationGame Boys: Gaming, Esports, and Young Men’s Mental Health
O’Gorman, K. M., Benakovic, R., & Fisher, K. (2024) · Springer eBooks · doi:10.1007/978-3-031-64053-7_4 · Cited by 5 · see citing works
Seetahul, Y., & Greitemeyer, T. (2024) · Social Psychological and Personality Science · doi:10.1177/19485506241273193 · Cited by 5 · see citing works
“However, a clear consensus remains elusive (de Vrieze, 2018; Mehta & Lakens, 2023), and some researchers argue that VVG research is hampered by methodological flaws and publication bias, leading to an overestimation of the actual effect (e.g., Hilgard et al., 2017).”
BackgroundFerguson, C. J., & Donnellan, M. B. (2017) · Journal of Youth and Adolescence · doi:10.1007/s10964-017-0768-3 · Cited by 5 · see citing works
Violent video games: content, attitudes, and norms
Andersson, A., & Milam, P. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09726-6 · Cited by 5 · see citing works
“Others have argued that the field suffers from a publication bias that favors finding an effect of VVGs on aggression (Ferguson, 2007; Ferguson & Kilburn, 2009; Hilgard et al., 2017).”
BackgroundADHD symptoms and callous‐unemotional traits as predictors of violent media use in adolescence
Rydell, A., & Brocki, K. C. (2020) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12693 · Cited by 5 · see citing works
Cultural literacy acquisition through video game environments of a digitally born generation.
Shliakhovchuk, O. (2019) · doi:10.4995/thesis/10251/130848 · Cited by 5 · see citing works
Hate and Extremism on Gaming Platforms
Amarasingam, A., & Kelley, D. (2024) · Routledge eBooks · doi:10.4324/9781003388371-7 · Cited by 4 · see citing works
Şiddet İçerikli Video Oyunları ve Saldırganlık İlişkisinin Gözden Geçirilmesi
Bozkuş-Demir, O. (2021) · Gelişim ve Psikoloji Dergisi · doi:10.51503/gpd.791346 · Cited by 4 · see citing works
Handsley, E., & Warburton, W. (2021) · Psychiatry Psychology and Law · doi:10.1080/13218719.2021.1904446 · Cited by 4 · see citing works
“….14, Ferguson, 2015; r ¼ .02 to .15, Hilgard et al., 2017; pooled estimates .04 to .17, Mathur & VanderWeele, 2019), and to the concerns of some researchers sceptical of a media violence–aggression link (e.g. Elson & Ferguson, 2014; Ferguson, 2015; Ferguson & Beresin, 2017; Hilgard et al., 2017).”
BackgroundInfluentialZhang, Q., & Gao, J. (2021) · Journal of Aggression Maltreatment & Trauma · doi:10.1080/10926771.2021.1984351 · Cited by 4 · see citing works
The Basic Needs in Games (BANG) Model of Video Game Play and Mental HealthPreprint
Ballou, N., & Deterding, S. (2023) · doi:10.31234/osf.io/6vedg · Cited by 4 · see citing works
How do video games affect mental health? A narrative review of 13 proposed mechanismsPreprint
Ballou, N., Hakman, T., Vuorre, M., Magnusson, K., & Przybylski, A. K. (2024) · doi:10.31234/osf.io/q2kxg · Cited by 4 · see citing works
Sánchez, M. A. A., & Ferguson, C. J. (2021) · Journal of Mass Violence Research · doi:10.53076/jmvr91931 · Cited by 3 · see citing works
- Authors
- Hilgard, J., Engelhardt, C. R., & Rouder, J. N.
- Affiliations
- University of Pennsylvania (US)
- University of Missouri (US)
- Venue
- Psychological Bulletin
- Published
- 2017-07
- Volume
- 143
- Issue
- 7
- Pages
- 757-774
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2633438608
- Semantic Scholar
- 2a45c07624c58a274ddadd219b25c3f2a23ac534
- Cited by
- 162
- References
- 64
- Citations by year
- 2026: 11 · 2025: 9 · 2024: 15 · 2023: 17 · 2022: 16 · 2021: 25 · 2020: 22 · 2019: 20 · 2018: 17 · 2017: 7 · 2016: 2
- FWCI
- 25.90
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Psychological Bulletin (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Funders
- Annenberg Public Policy Center
- National Science Foundation (BCS-1240359 and SES-102408)
- Topics
- Social Sciences › Psychology › Social Psychology › Bullying, Victimization, and Aggression
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- violent video games, aggression, aggressive affect, publication bias, research bias, trim and fill, methodological quality, preregistration, reanalysis
- MeSH terms
- Aggression, Humans, Violence, Meta-Analysis as Topic, Publication Bias, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
