The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Overstated evidence for short-term effects of violent games on affect and behavior: A reanalysis of Anderson et al. (2010).
Open accessAbstract
Violent video games are theorized to be a significant cause of aggressive thoughts, feelings, and behaviors. Important evidence for this claim comes from a large meta-analysis by Anderson and colleagues (2010), who found effects of violent games in experimental, cross-sectional, and longitudinal research. In that meta-analysis, the authors argued that there is little publication or analytic bias in the literature, an argument supported by their use of the trim-and-fill procedure. In the present manuscript, we reexamine their meta-analysis using a wider array of techniques for detecting bias and adjusting effect sizes. Our conclusions differ from those of Anderson and colleagues in 3 salient ways. First, we detect substantial publication bias in experimental research on the effects of violent games on aggressive affect and aggressive behavior. Second, after adjustment for bias, the effects of violent games on aggressive behavior in experimental research are estimated as being very small, and estimates of effects on aggressive affect are much reduced. In contrast, the cross-sectional literature finds correlations that appear largely unbiased. Third, experiments meeting the original authors' criteria for methodological quality do not yield larger adjusted effects than other experiments, but instead yield larger indications of bias, indicating that perhaps they were selected for significance. We outline future directions for stronger experimental research. The results indicate th…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“this analysis also suggested that studies originally deemed to be following best practices showed particularly strong evidence of publication bias.”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
In the citing paper
“Upon adjusting for publication bias, the observed aggregate effect sizes relating gaming to aggression were smaller than those originally presented.”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
In the citing paper
“The existing literature suggests that the effect linking violent video game play to aggressive behaviour in cross-sectional research (n = 29 113) is approximately r = 0.21 (95% CI = 0.20–0.22).”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
In the citing paper
“Given this effect, r = 0.01 (95% CI = −0.08 to 0.10), did not overlap with, and was clearly inferior to, r = 0.21 (95% CI = 0.20–0.22)”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
In the citing paper
“Their analysis detected the presence of publication bias not uncovered in the original reporting of the data.”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
In the citing paper
“because we prespecified how we would interpret effects larger, smaller or falling in the same range as estimates from meta-analysis [32]”
From Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report (2019) · cites this paper as [32]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
James, W. (1891) · Personality Culture Society · doi:10.1037/11059-000 · Cited by 25,876
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The unbearable automaticity of being
Bargh, J. A., & Chartrand, T. L. (1999) · American Psychologist · doi:10.1037/0003-066x.54.7.462 · Cited by 3,699
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein, H. R., & Saleem, M. (2010) · Psychological Bulletin · doi:10.1037/a0018251 · Cited by 2,048
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life.
Anderson, C. A., & Dill, K. E. (2000) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.78.4.772 · Cited by 1,955
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Meta-analysis of individual participant data: rationale, conduct, and reporting
Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010) · BMJ · doi:10.1136/bmj.c221 · Cited by 1,701
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
A Multilab Preregistered Replication of the Ego-Depletion Effect
Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H. J. E. M., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brand, M., Brewer, G., Bruyneel, S. D., et al. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616652873 · Cited by 1,097
Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified
Higgins, J. P. T. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn204 · Cited by 995
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751
p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717
Violent Video Games and Hostile Expectations: A Test of the General Aggression Model
Bushman, B. J., & Anderson, C. A. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/014616702237649 · Cited by 670
(2009) · Cited by 670
Video Games Do Affect Social Outcomes
Greitemeyer, T., & Mügge, D. O. (2014) · Personality and Social Psychology Bulletin · doi:10.1177/0146167213520459 · Cited by 652
Ferguson, C. J. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615592234 · Cited by 524
Evidence for publication bias in video game violence effects literature: A meta-analytic review
Ferguson, C. J. (2007) · Aggression and Violent Behavior · doi:10.1016/j.avb.2007.01.001 · Cited by 398
Carter, E. C., & McCullough, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00823 · Cited by 381
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373
A general linear model for estimating effect size in the presence of publication bias
Vevea, J. L., & Hedges, L. (1995) · Psychometrika · doi:10.1007/bf02294384 · Cited by 367
The Public Health Risks of Media Violence: A Meta-Analytic Review
Ferguson, C. J., & Kilburn, J. C. (2009) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2008.11.033 · Cited by 330
Ferguson, C. J., & Kilburn, J. (2010) · Psychological Bulletin · doi:10.1037/a0018566 · Cited by 311
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265
Competence-impeding electronic games and players' aggressive feelings, thoughts, and behaviors
Przybylski, A. K., Deci, E. L., Rigby, C. S., & Ryan, R. M. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0034820 · Cited by 263
Priming Intelligent Behavior: An Elusive Phenomenon
Shanks, D. R., Newell, B. R., Lee, E. H., Balakrishnan, D., Ekelund, L., Cenac, Z., Kavvadia, F., & Moore, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0056515 · Cited by 263
The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles
O’Boyle, E. H., Banks, G. C., & Gonzalez-Mulé, E. (2017) · Journal of Management · doi:10.1177/0149206314527133 · Cited by 246
On the reproducibility of meta-analyses: six practical recommendations
Lakens, D., Hilgard, J., & Staaks, J. P. C. (2016) · BMC Psychology · doi:10.1186/s40359-016-0126-3 · Cited by 238
Self-citationElson, M., & Ferguson, C. J. (2014) · European Psychologist · doi:10.1027/1016-9040/a000147 · Cited by 237
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219
Ballard, M. E., & Wiest, J. (1996) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1996.tb02740.x · Cited by 196
Willoughby, T., Adachi, P. J. C., & Good, M. (2011) · Developmental Psychology · doi:10.1037/a0026046 · Cited by 178
Elson, M., Mohseni, M. R., Breuer, J., Scharkow, M., & Quandt, T. (2014) · Psychological Assessment · doi:10.1037/a0035569 · Cited by 159
The effect of horizontal eye movements on free recall: a preregistered adversarial collaboration.
Matzke, D., Nieuwenhuis, S., van Rijn, H., Slagter, H., van der Molen, M. W., & Wagenmakers, E. (2015) · Journal of experimental psychology. General · doi:10.1037/xge0000038 · Cited by 148
Huesmann, L. R. (2010) · Psychological Bulletin · doi:10.1037/a0018567 · Cited by 113
Does Cleanliness Influence Moral Judgments?
Johnson, D. J., Cheung, F., & Donnellan, M. B. (2014) · Social Psychology · doi:10.1027/1864-9335/a000186 · Cited by 105
Bushman, B. J., Rothstein, H. R., & Anderson, C. A. (2010) · Psychological Bulletin · doi:10.1037/a0018718 · Cited by 104
Graybill, D. F., Kirsch, J. R., & Esselman, E. D. (1985) · Child study journal · Cited by 99
Violent video games and physical aggression: Evidence for a selection effect among adolescents.
Breuer, J., Vogelgesang, J., Quandt, T., & Festl, R. (2015) · Psychology of Popular Media Culture · doi:10.1037/ppm0000035 · Cited by 98
An Empirical Investigation of Partial Effect Sizes in Meta-Analysis of Correlational Data
Aloe, A. M. (2013) · The Journal of General Psychology · doi:10.1080/00221309.2013.853021 · Cited by 92
Angry Birds, Angry Children, and Angry Meta-Analysts
Furuya‐Kanamori, L., & Doi, S. A. R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616635599 · Cited by 85
Boxer, P., Groves, C. L., & Docherty, M. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615592239 · Cited by 68
Kneer, J., Elson, M., & Knapp, F. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.07.034 · Cited by 66
A Bayesian approach to mitigation of publication bias
Guan, M., & Vandekerckhove, J. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0868-6 · Cited by 63
DeWall, C. N., & Bushman, B. J. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.05.003 · Cited by 61
Etchells, P. J., Gage, S. H., Rutherford, A. D., & Munafo, M. R. (2016) · PLoS ONE · doi:10.1371/journal.pone.0147732 · Cited by 61
Extreme between-study homogeneity in meta-analyses could offer useful insights
Ioannidis, J. P. A., Trikalinos, T. A., & Zintzaras, E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.013 · Cited by 58
Engelhardt, C. R., Mazurek, M. O., Hilgard, J., Rouder, J. N., & Bartholow, B. D. (2015) · Psychological Science · doi:10.1177/0956797615583038 · Cited by 55
Self-citationPanee, C. D., & Ballard, M. E. (2002) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2002.tb02751.x · Cited by 52
Tear, M. J., & Nielsen, M. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.09.002 · Cited by 48
Elson, M., Breuer, J., LOOY, J. V., Kneer, J., & Quandt, T. (2013) · Psychology of Popular Media Culture · doi:10.1037/ppm0000010 · Cited by 47
Engelhardt, C. R., Bartholow, B. D., & Saults, J. S. (2011) · Aggressive Behavior · doi:10.1002/ab.20411 · Cited by 43
Self-citationSigurðsson, J. F., Gudjonsson, G. H., Bragason, A. V., Kristjansdottir, E., & Sigfúsdóttir, I. D. (2006) · Personality and Individual Differences · doi:10.1016/j.paid.2006.02.006 · Cited by 39
Digital games in laboratory experiments: Controlling a complex stimulus through modding.
Elson, M., & Quandt, T. (2014) · Psychology of Popular Media Culture · doi:10.1037/ppm0000033 · Cited by 35
The consistency test does not–and cannot–deliver what is advertised: A comment on Francis (2013)
Morey, R. D. (2013) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2013.03.004 · Cited by 26
Acute exposure to difficult (but not violent) video games dysregulates cognitive control
Engelhardt, C. R., Hilgard, J., & Bartholow, B. D. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.11.089 · Cited by 23
Self-citationClose replication attempts of the heat priming-hostile perception effect
McCarthy, R. J. (2014) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2014.04.014 · Cited by 14
Violent Media Effects on Aggression: A Commentary from a Cross‐Cultural Perspective
Krahé, B. (2016) · Analyses of Social Issues and Public Policy · doi:10.1111/asap.12107 · Cited by 5
Engelhardt, C. R., Mazurek, M. O., Hilgard, J., Rouder, J. N., & Bartholow, B. D. (2014) · Cited by 1
Self-citation
Cited by
Sorted by citations
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588
Self-citationMedia Use Is Linked to Lower Psychological Well-Being: Evidence from Three Datasets
Twenge, J. M., & Campbell, W. K. (2019) · Psychiatric Quarterly · doi:10.1007/s11126-019-09630-7 · Cited by 429
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354
“…could be explained away by publication bias; other researchers suggested that publication bias might largely explain these results (Ferguson and Kilburn, 2010; Hilgard et al., 2017), whereas the original authors argued that the results were in fact robust to publication bias (Kepes et al., 2017).)”
ResultWhen and How Video Games Can Be Good: A Review of the Positive Effects of Video Games on Well-Being
Halbrook, Y. J., O’Donnell, A. T., & Msetfi, R. M. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619863807 · Cited by 263
Beyond Screen Time: Identity Development in the Digital Age
Granic, I., Morita, H., & Scholten, H. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1820214 · Cited by 237
Metadata
- Authors
- Hilgard, J., Engelhardt, C. R., & Rouder, J. N.
- Affiliations
- University of Pennsylvania (US)
- University of Missouri (US)
- Venue
- Psychological Bulletin
- Published
- 2017-07
- Volume
- 143
- Issue
- 7
- Pages
- 757-774
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2633438608
- Semantic Scholar
- 2a45c07624c58a274ddadd219b25c3f2a23ac534
- Cited by
- 162
- References
- 64
- Citations by year
- 2026: 11 · 2025: 9 · 2024: 15 · 2023: 17 · 2022: 16 · 2021: 25 · 2020: 22 · 2019: 20 · 2018: 17 · 2017: 7
- FWCI
- 25.90
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Psychological Bulletin (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Funders
- Annenberg Public Policy Center
- National Science Foundation (BCS-1240359 and SES-102408)
- Topics
- Social Sciences › Psychology › Social Psychology › Bullying, Victimization, and Aggression
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- violent video games, aggression, aggressive affect, publication bias, research bias, trim and fill, methodological quality, preregistration, reanalysis
- MeSH terms
- Aggression, Humans, Violence, Meta-Analysis as Topic, Publication Bias, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
