Violent video game engagement is not associated with adolescents' aggressive behaviour: evidence from a registered report

In the paper

“the competitive reaction time task (CRTT; [34]), has been used in more than 125 published papers, and surprisingly, task scores have been quantified in more than 155 different ways in this literature [44].”

What the sources say

• ?Could not verify

  In the citing sentence, this source is cited for identifying the competitive reaction time task; the claims about more than 125 papers and 155 scoring methods are attached to a different citation. Searches for the task and those numbers found no matching passage in our copy. Study 3 Methods instead describes a tangram-puzzle task, so this source does not verify the identification. (LLM, conf=0.76)

  cites [34] Gentile DA et al. 2009 The effects of prosocial video games on prosocial behaviors: international evidence from correlational, longitudinal, and experimental studies. Pers. Soc. Psychol. Bull. 35, 752– 763. (10.1177/0146167209333045) 19321812 PMC2678173 sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [44] Elson M 2017 FlexibleMeasures.com: competitive reaction time task (CRTT). [cited 3 August 2016]. See http://crtt.flexiblemeasures.com/index.php?menu=quantifications.

In the paper

“the present study examined the fundamental dynamic of concern in this subfield through a purely confirmatory lens [47] following a registered reports protocol [48].”

What the sources say

• ?Not checked — no full text of the sourcecites [47] McCarthy RJ, Coley SL, Wagner MF, Zengel B, Basham A 2016 Does playing video games with violent content temporarily increase aggressive inclinations? A pre-registered experimental study. J. Exp. Soc. Psychol. 67, 13– 19. (10.1016/j.jesp.2015.10.009) source
• Supported

  The citation to this source, [47], is attached to the study’s “purely confirmatory lens,” not to its Registered Reports protocol, which has a separate citation [48]. In “Reporting and dissemination,” the source distinguishes data-independent confirmatory research for testing hypotheses from data-contingent exploratory research. It provides the methodological basis for that description; it does not independently verify how the citing study was conducted. (LLM, conf=0.86)

  cites [48] Munafò MR et al. 2017 A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (10.1038/s41562-016-0021) PMC7610724 33954258 source

In the paper

“The conduct problems subscale of the SDQ was the study's outcome variable, as it has been extensively used to measure aggressive behaviour and interpersonal aggression across a wide range of cultures in community [58], academic [59] and clinical [60] settings.”

What the sources say

• Partly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites [58] Goodman R, Ford T, Simmons H, Gatward R, Meltzer H 2000 Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br. J. Psychiatry J. Ment. Sci. 177, 534– 539. (10.1192/bjp.177.6.534) 11102329 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [59] Matsuishi T et al. 2008 Scale properties of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ): a study of infant and school children in community samples. Brain Dev. 30, 410– 415. (10.1016/j.braindev.2007.12.003) 18226867 source
• ?Not checked — no full text of the sourcecites [60] Becker A, Hagenberg N, Roessner V, Woerner W, Rothenberger A 2004 Evaluation of the self-reported SDQ in a clinical setting: do self-reports tell us more than ratings by adult informants? Eur. Child Adolesc. Psychiatry 13(Suppl. 2), II17– II24. 15243782 source

In the paper

“Indeed, statistical significance does not necessitate a subjective state that humans can distinguish on a personal level [77,78].”

What the sources say

• ?Could not verify

  In the discussion of absolute judgments (p. 85), Miller describes limits on how many stimulus intensities people can discriminate. The passages found by searching for statistical significance and subjective distinguishability do not address whether a statistically significant result corresponds to a personally distinguishable subjective state. (LLM, conf=0.73)

  cites [77] Miller GA 1956 The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63, 81– 97. (10.1037/h0043158) 13310704 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [78] Norman GR, Sloan JA, Wyrwich KW 2003 Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med. Care 41, 582– 592. (10.1097/01.MLR.0000062554.74615.4C) 12719681 source

In the paper

“Similarly, past research indicates that levels of daily video game play [2], violent game play [66] and aggressive behaviour should all be higher in male adolescents [58].”

What the sources say

• Supported

  The citation to this paper is attached to daily video game play, not violent game play or aggressive behaviour. The Exploratory analyses (p. 6) report that boys spent far more time gaming than girls on both weekdays and weekend days. (LLM, conf=0.93)

  cites [2] Przybylski AK, Weinstein N 2017 A large-scale test of the goldilocks hypothesis: quantifying the relations between digital-screen use and the mental well-being of adolescents. Psychol. Sci. 28, 204– 215. (10.1177/0956797616678438) 28085574 source
• ?Not checked — no full text of the sourcecites [66] Kutner L, Olson CK 2008 Grand theft childhood: the surprising truth about violent video games and what parents can do. New York, NY: Simon & Schuster.
• ?Could not verify

  The citation [58] is attached to the expectation that aggressive behaviour is higher in male adolescents. The Sample section reports the participants’ sex distribution, and Characteristics of ‘false positives’ says SDQ false positives were more likely to be male; neither compares aggressive behaviour by sex among adolescents. Searches for aggression and sex differences found no passage addressing that comparison. (LLM, conf=0.76)

  cites [58] Goodman R, Ford T, Simmons H, Gatward R, Meltzer H 2000 Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br. J. Psychiatry J. Ment. Sci. 177, 534– 539. (10.1192/bjp.177.6.534) 11102329 sourceExperimental — accuracy not yet measured

In the paper

“There is a growing literature on the links between gaming and prosocial behaviour and we decided the current data could speak to this literature as well [29,73,74].”

What the sources say

• ?Not checked — no full text of the sourcecites [29] Ferguson CJ 2015 Do angry birds make for angry children? A meta-analysis of video game influences on children's and adolescents’ aggression, mental health, prosocial behavior, and academic performance. Perspect. Psychol. Sci. 10, 646– 666. (10.1177/1745691615592234) 26386002 source
• ?Not checked — no full text of the sourcecites [73] Breuer J, Velez J, Bowman N, Wulf T, Bente G 2017 ‘Drive the lane; together, hard!’: an examination of the effects of supportive coplaying and task difficulty on prosocial behavior. J. Media Psychol. 29, 31– 41. (10.1027/1864-1105/a000209) source
• Supported

  The Introduction describes multiple studies linking video-game play with prosocial outcomes, including cooperation, helping, and rewarding others. It also notes that this literature is comparatively small and that few studies directly contrast violent and prosocial games; those qualifications do not negate the broad claim that a literature exists. (LLM, conf=0.86)

  cites [74] Tear MJ, Nielsen M 2013 Failure to demonstrate that playing violent video games diminishes prosocial behavior. PLoS ONE 8, e68382 (10.1371/journal.pone.0068382) 23844191 PMC3700923 source

In the paper

“we tested relations with gender, and observed higher levels of overall gaming, t764 = 3.71, p < 0.001, violent gaming, t764 = 4.09, p < 0.001 and aggressive behaviour, t999 = 2.98, p < 0.001, in adolescent boys as compared to adolescent girls.”

In the paper

“subjective game engagement was positively correlated with overall levels of game engagement, any violent game engagement and overall levels of violent game engagement (rs = 0.18–0.43)”

In the paper

“In line with expectations, aggressive behaviour was positively associated with all of the observed variables (|r|s = 0.08–0.62)”

In the paper

“Results from this model showed that neither the linear (p = 0.402) nor the parabolic (p = 0.624) predictors were statistically significant.”

In the paper

“Sensitivity and exploratory analyses indicated these null effects extended across multiple operationalizations of violent game engagement and when the focus was on another behavioural outcome, namely, prosocial behaviour.”

In the paper

“We expected and found that responses to this item were positively correlated with participant estimates of the time they spent playing violent games on a typical day, r768 = 0.47, p < 0.001.”

In the paper

“we found participants were moderately engaged in violent games, devoting an average of two hours to them on a typical day (M = 2.07, s.d. = 2.30).”

In the paper

“Results from this model, presented in table 3, indicated both the linear (p = 0.899) and parabolic (p = 0.851) predictors were not statistically significant.”

In the paper

“Perhaps unsurprisingly then, results, presented in table 4, mirrored those observed in both the primary and sensitivity analyses.”

In the paper

“This was not possible because the parabolic term was not significant (p = 0.624), and as such we did not have an empirical basis to calculate the local extrema as planned.”

In the paper

“For example, our positive control check question, a simple 5-point Likert scale that asked participants to rate their own intensity of engagement with games happened to be positively correlated (r = 0.25) to aggressive behaviour.”

In the paper

“A correlation analysis showed that participants' recent violent game time, using PEGI and ESRB ratings, were highly interrelated (r = 0.80, p < 0.001).”

In the paper

“A total of 48.8% of female participants and 68.0% of males played at least one violent game in the past month.”

In the paper

“Violent game engagement was not a statistically significant linear (p = 0.98) or nonlinear (p = 0.07) predictor of aggressive behaviour.”

In the paper

“A participant profile of adolescents aged 14 and 15 years living in England, Scotland and Wales was built using geographical and demographic factors based on 2011 United Kingdom Census data. Geographical region, household socioeconomic class, participant age and gender were considered for quotas to match the census data and sampling continued until these quotas were attained.”

In the paper

“Given this, we controlled for variability in gender in our hypothesis testing as planned”

In the paper

“Upon adjusting for publication bias, the observed aggregate effect sizes relating gaming to aggression were smaller than those originally presented.”

What the sources say

• ?Not checked — no full text of the sourcecites [32] Hilgard J, Engelhardt CR, Rouder JN 2017 Overstated evidence for short-term effects of violent games on affect and behavior: a reanalysis of Anderson et al. 2010. Psychol. Bull. 143, 757– 774. (10.1037/bul0000074) 28639810 source

In the paper

“Individuals' scores on the five-item conduct problems subscale of the SDQ ranged from 5 to 15 (M = 6.93, s.d. = 2.20, α = 0.76)”

In the paper

“Following a preregistered analysis plan, multiple regression analyses tested the hypothesis that recent violent game play is linearly and positively related to carer assessments of aggressive behaviour.”

In the paper

“The study underwent ethical review and received approval from the Central University Ethics Committee of the University of Oxford (C1A17023)”

In the paper

“A handful of preregistered studies have rigorously tested the links between violent game play and human aggression [47], and do not detect an effect of brief exposure to violent gaming on aggression in the laboratory.”

What the sources say

• ?Not checked — no full text of the sourcecites [47] McCarthy RJ, Coley SL, Wagner MF, Zengel B, Basham A 2016 Does playing video games with violent content temporarily increase aggressive inclinations? A pre-registered experimental study. J. Exp. Soc. Psychol. 67, 13– 19. (10.1016/j.jesp.2015.10.009) source

In the paper

“Their analysis detected the presence of publication bias not uncovered in the original reporting of the data.”

What the sources say

• ?Not checked — no full text of the sourcecites [32] Hilgard J, Engelhardt CR, Rouder JN 2017 Overstated evidence for short-term effects of violent games on affect and behavior: a reanalysis of Anderson et al. 2010. Psychol. Bull. 143, 757– 774. (10.1037/bul0000074) 28639810 source

In the paper

“Further, the explanatory variable, violent video game play, reflected the level of exposure young people have to violent games by combining objective assessments of both time spent playing and violent content (official game ratings) across a broad range of popular games.”

In the paper

“Given this effect, r = 0.01 (95% CI = −0.08 to 0.10), did not overlap with, and was clearly inferior to, r = 0.21 (95% CI = 0.20–0.22)”

What the sources say

• ?Not checked — no full text of the sourcecites [32] Hilgard J, Engelhardt CR, Rouder JN 2017 Overstated evidence for short-term effects of violent games on affect and behavior: a reanalysis of Anderson et al. 2010. Psychol. Bull. 143, 757– 774. (10.1037/bul0000074) 28639810 source

In the paper

“all study materials are available for download on the Open Science Framework [57].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [57] Przybylski AK, Weinstein N 2017 Study materials. See https://osf.io/rkw6z/.

In the paper

“This information was then used by a coder who was blind to the purpose of the study to find the game's entry on the Pan European Game Information (PEGI) website for confirmatory hypothesis testing.”

In the paper

“this analysis also suggested that studies originally deemed to be following best practices showed particularly strong evidence of publication bias.”

What the sources say

• ?Not checked — no full text of the sourcecites [32] Hilgard J, Engelhardt CR, Rouder JN 2017 Overstated evidence for short-term effects of violent games on affect and behavior: a reanalysis of Anderson et al. 2010. Psychol. Bull. 143, 757– 774. (10.1037/bul0000074) 28639810 source

In the paper

“the naive and corrected estimated effect sizes relating violent video game play to self-reported aggression extracted from this meta-analysis across a total of 37 studies (n = 29 113) were relatively consistent and small to medium [33] (r = 0.21; 95% CI = 0.20–0.22) in size.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [33] Cohen J 1988 Statistical power analysis for the behavioral sciences, 2nd edn Hillsdale, NJ: Routledge.

In the paper

“This flexibility, described as part of the ‘garden of forking paths problem’ increases the chance of false-positive results and serves to reduce our confidence in the inferences linking gaming to aggression [37,38].”

What the sources say

• ?Not checked — no full text of the sourcecites [37] Simmons JP, Nelson LD, Simonsohn U 2011 False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359– 1366. (10.1177/0956797611417632) 22006061 source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [38] Gelman A, Loken E 2013 The garden of forking paths: why multiple comparisons can be a problem, even when there is no ‘fishing expedition’ or ‘p-hacking’ and the research hypothesis was posited ahead of time. New York, NY: Dep Stat Columbia Univ.

In the paper

“we concluded this observed effect relating violent gaming to aggressive behaviour was both statistically and practically insignificant.”

In the paper

“Findings can be interpreted with greater confidence because these standards were set for type 1 and 2 error control and the sample was sufficiently large for a fair and sensitive test of the null and alternative hypotheses.”

In the paper

“Scores on this five-item subscale were summed and ranged from 5 to 15 (M = 12.08, s.d. = 2.29, α = 0.77).”

In the paper

“Finally, in this study, we draw general inferences about gaming effects in a general way across the population as a whole.”

In the paper

“because we prespecified how we would interpret effects larger, smaller or falling in the same range as estimates from meta-analysis [32]”

What the sources say

• ?Not checked — no full text of the sourcecites [32] Hilgard J, Engelhardt CR, Rouder JN 2017 Overstated evidence for short-term effects of violent games on affect and behavior: a reanalysis of Anderson et al. 2010. Psychol. Bull. 143, 757– 774. (10.1037/bul0000074) 28639810 source

In the paper

“Our study, then, contributes to this literature base, as we prespecified exactly how we would operationalize key variables before conducting the study.”

In the paper

“our criterion variable—the Strengths and Difficulties Questionnaire—was completed by carers (not adolescent participants)”

In the paper

“We believe the current work is the second study in the area to set the number of observations based on an a priori power analysis [47], and the first to specify a minimum effect size of theoretical interest.”

What the sources say

• ?Not checked — no full text of the sourcecites [47] McCarthy RJ, Coley SL, Wagner MF, Zengel B, Basham A 2016 Does playing video games with violent content temporarily increase aggressive inclinations? A pre-registered experimental study. J. Exp. Soc. Psychol. 67, 13– 19. (10.1016/j.jesp.2015.10.009) source

In the paper

“Our data supported this decision, as the reliability of the scale was relatively high and in line with previous SDQ work [59].”

What the sources say

• ?Not checked — no full text of the sourcecites [59] Matsuishi T et al. 2008 Scale properties of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ): a study of infant and school children in community samples. Brain Dev. 30, 410– 415. (10.1016/j.braindev.2007.12.003) 18226867 source

In the paper

“before conducting the study, we set both a criterion for statistical significance and an a priori threshold informed by the extant literature as a criterion for practical significance.”

In the paper

“This pattern of findings further suggests that links reported in the literature might be influenced by publication bias, selective reporting, or an artefact of unobserved or hidden moderators, as has been previously suspected [45,78].”

What the sources say

• ?Not checked — no full text of the sourcecites [45] Anderson CA, Dill KE 2000 Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. J. Pers. Soc. Psychol. 78, 772– 790. (10.1037/0022-3514.78.4.772) 10794380 source
• ?Not checked — no full text of the sourcecites [78] Norman GR, Sloan JA, Wyrwich KW 2003 Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med. Care 41, 582– 592. (10.1097/01.MLR.0000062554.74615.4C) 12719681 source

In the paper

“Second, the present study is based on cross-sectional data.”

In the paper

“This is among the first studies to test the effects of violent gaming on human aggression using a preregistered hypothesis-testing framework and the first to do so following the registered reports protocol.”

In the paper

“a scientific literature sharply divided on the effects of violent video games [29]”

What the sources say

• ?Not checked — no full text of the sourcecites [29] Ferguson CJ 2015 Do angry birds make for angry children? A meta-analysis of video game influences on children's and adolescents’ aggression, mental health, prosocial behavior, and academic performance. Perspect. Psychol. Sci. 10, 646– 666. (10.1177/1745691615592234) 26386002 source

In the paper

“First, our work is based entirely on self-report data.”