Evaluating the quality of evidence for gaming disorder: A summary of systematic reviews of associations between gaming disorder and depression or anxiety
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Gaming disorder has been described as an urgent public health problem and has garnered many systematic reviews of its associations with other health conditions. However, review methodology can contribute to bias in the conclusions, leading to research, policy, and patient care that are not truly evidence-based. This study followed a pre-registered protocol (PROSPERO 2018 CRD42018090651) with the objective of identifying reliable and methodologically-rigorous systematic reviews that examine the associations between gaming disorder and depression or anxiety in any population. We searched PubMed and PsycInfo for published systematic reviews and the gray literature for unpublished systematic reviews as of June 24, 2020. Reviews were classified as reliable according to several quality criteria, such as whether they conducted a risk of bias assessment of studies and whether they clearly described how outcomes from each study were selected. We assessed possible selective outcome reporting among the reviews. Seven reviews that included a total of 196 studies met inclusion criteria. The overall number of participants was not calculable because not all reviews reported these data. All reviews specified eligibility criteria for studies, but not for outcomes within studies. Only one review assessed risk of bias. Evidence of selective outcome reporting was found in all reviews-only one review incorporated any of the null findings from studies it included. Thus, none were classified as reliable according to prespecified quality criteria. Systematic reviews related to gaming disorder do not meet methodological standards. As clinical and policy decisions are heavily reliant on reliable, accurate, and unbiased evidence synthesis; researchers, clinicians, and policymakers should consider the implications of selective outcome reporting. Limitations of the current summary include using counts of associations and restricting to systematic reviews published in English. Systematic reviewers should follow established guidelines for review conduct and transparent reporting to ensure evidence about technology use disorders is reliable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Pathological Video Game Use Among Youths: A Two-Year Longitudinal Study
Gentile, D. A., Choo, H., Liau, A., Sim, T., Li, D., Fung, D., & Khoo, A. (2011) · Pediatrics · doi:10.1542/peds.2010-1353 · Cited by 1,314
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Internet Gaming Addiction: A Systematic Review of Empirical Research
Kuss, D. J., & Griffiths, M. D. (2012) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-011-9318-5 · Cited by 1,113
The ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838
Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768 · cited 2×
“Recent decades have witnessed a surge in the number of systematic reviews conducted [8].”
BackgroundMihara, S., & Higuchi, S. (2017) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.12532 · Cited by 611 · cited 2×
“Most reviews discussed limitations at the study, outcome, and review level, but two reviews did not discuss limitations systematically [27,29].”
BackgroundThe relationships between behavioral addictions and the five-factor model of personality
Andreassen, C. S., Griffiths, M. D., Gjertsen, S. R., Krossbakken, E., Kvam, S., & Pallesen, S. (2013) · Journal of Behavioral Addictions · doi:10.1556/jba.2.2013.003 · Cited by 589
External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569 · cited 1×
“This definition, in turn, was informed by items identified from the Critical Appraisal Skills Programme (CASP), A Measurement Tool to Assess systematic Reviews (AMSTAR), and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) tools [9,22,23].”
MethodGaming disorder: Its delineation as an important condition for diagnosis, management, and prevention
Saunders, J. B., Hao, W., Long, J., King, D. L., Mann, K., Fauth-Bühler, M., Rumpf, H.-J., Bowden-Jones, H., Rahimi-Movaghar, A., Chung, T., et al. (2017) · Journal of Behavioral Addictions · doi:10.1556/2006.6.2017.039 · Cited by 489
González-Bueso, V., Santamaría, J., Fernández, D., Merino, L., Montero, E., & Ribas, J. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15040668 · Cited by 487
Rehbein, F., Psych, G., Kleimann, M., Mediasci, G., & Mößle, T. (2010) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2009.0227 · Cited by 480
Brunborg, G. S., Mentzoni, R. A., & Frøyland, L. R. (2014) · Journal of Behavioral Addictions · doi:10.1556/jba.3.2014.002 · Cited by 445 · cited 1×
“Only one study reporting an association between gaming disorder and depression examined both cross-sectional and longitudinal associations, and the results were inconsistent [38].”
ResultOnline gaming addiction in children and adolescents: A review of empirical research
Kuss, D. J., & Griffiths, M. D. (2012) · Journal of Behavioral Addictions · doi:10.1556/jba.1.2012.1.1 · Cited by 418 · cited 1×
“However, the three reviews that included this study and reported results for depression all reported them as positive [30,31,33].”
BackgroundQuality of Reporting of Randomized Trials as a Measure of Methodologic Quality
Huwiler-Müntener, K. (2002) · JAMA · doi:10.1001/jama.287.21.2801 · Cited by 397
Király, O., Griffiths, M. D., Urbán, R., Farkas, J., Kökönyei, G., Elekes, Z., Tamás, D., & Demetrovics, Z. (2014) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2014.0475 · Cited by 377
Walther, B., Morgenstern, M., & Hanewinkel, R. (2012) · European Addiction Research · doi:10.1159/000335662 · Cited by 368
Manifesto for a European research network into Problematic Usage of the Internet
Fineberg, N., Demetrovics, Z., Stein, D., Ioannidis, K., Potenza, M., Grünblatt, E., Brand, M., Billieux, J., Carmi, L., King, D., et al. (2018) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2018.08.004 · Cited by 334 · cited 1×
“From a public health perspective, many forms of Internet use—not just gaming—continue to be recognized as potentially problematic, as evidenced by a recently-funded international research collaborative on problematic Internet use [7].”
BackgroundHan, D. H., Hwang, J. W., & Renshaw, P. F. (2010) · Experimental and Clinical Psychopharmacology · doi:10.1037/a0020023 · Cited by 324
Systematic review adherence to methodological or reporting quality
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0527-2 · Cited by 322
Rumpf, H.-J., Achab, S., Billieux, J., Bowden-Jones, H., Carragher, N., Demetrovics, Z., Higuchi, S., King, D. L., Mann, K., Potenza, M., et al. (2018) · Journal of Behavioral Addictions · doi:10.1556/2006.7.2018.59 · Cited by 300 · cited 2×
“Many recent commentaries on the need for a diagnosis of gaming disorder use terms like “Internet addiction or Gaming disorder” [6], “Internet-related disorders including gaming disorder” [4], and “Internet addiction including gaming addiction” [5], pointing to the persistent overlap in measurement of these problematic…”
BackgroundPage, M. J., McKenzie, J. E., Kirkham, J., Dwan, K., Kramer, S., Green, S., & Forbes, A. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000035.pub2 · Cited by 281 · cited 1×
“Selective reporting of outcomes can be hard to detect, and further research into the impact of selective inclusion of results in reviews is needed to advance the understanding of this form of bias on evidence synthesis [77,78].”
BackgroundProblematic gaming behaviour and health-related outcomes: A systematic review and meta-analysis
Männikkö, N., Ruotsalainen, H., Miettunen, J., Pontes, H. M., & Kääriäinen, M. (2020) · Journal of Health Psychology · doi:10.1177/1359105317740414 · Cited by 243 · cited 6×
“that did not measure gaming disorder, and the lack of systematic assessment of bias (except for one review [30]), we determined that review conclusions were not supported by the evidence from included studies.”
BackgroundResultInfluentialLiu, M., & Peng, W. (2009) · Computers in Human Behavior · doi:10.1016/j.chb.2009.06.002 · Cited by 221
Mayo-Wilson, E., Fusco, N., Li, T., Hong, H., Canner, J. K., & Dickersin, K. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.007 · Cited by 131 · cited 3×
“However, research has shown that reviews in some fields provide low-quality evidence, are unreliable, and can be sources of bias themselves [8,10,11].”
BackgroundColder Carras, M., Kalbarczyk, A., Wells, K., Banks, J., Kowert, R., Gillespie, C., & Latkin, C. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.08.044 · Cited by 128
Pápay, O., Urbán, R., Griffiths, M. D., Nagygyörgy, K., Farkas, J., Kökönyei, G., Felvinczi, K., Oláh, A., Elekes, Z., & Demetrovics, Z. (2013) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2012.0484 · Cited by 118
Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy
Mayo-Wilson, E., Li, T., Fusco, N., Bertizzolo, L., Canner, J. K., Cowley, T., Doshi, P., Ehmsen, J., Gresham, G., Guo, N., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.07.014 · Cited by 108
Higuchi, S., Nakayama, H., Mihara, S., Maezono, M., Kitayuguchi, T., & Hashimoto, T. (2017) · Journal of Behavioral Addictions · doi:10.1556/2006.6.2017.049 · Cited by 67
Cited by
Video game play is positively correlated with well-being
Johannes, N., Vuorre, M., & Przybylski, A. K. (2021) · Royal Society Open Science · doi:10.1098/rsos.202049 · Cited by 330
Future of Mental Health in the Metaverse
Usmani, S. S., Sharath, M., & Mehendale, M. (2022) · General Psychiatry · doi:10.1136/gpsych-2022-100825 · Cited by 262
Subtyping treatment-seeking gaming disorder patients
Granero, R., Fernández-Aranda, F., Castro-Calvo, J., Billieux, J., Valero-Solís, S., Mora-Maltas, B., Rivas-Pérez, S., Valenciano-Mendoza, E., del Pino-Gutiérrez, A., Gómez-Peña, M., et al. (2021) · Addictive Behaviors · doi:10.1016/j.addbeh.2021.107086 · Cited by 23
Throuvala, M. A., Griffiths, M. D., Rennoldson, M., & Kuss, D. J. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.648512 · Cited by 19
“…et al., 2020), with academic debates arguing the thresholds of normative and problematic behaviours (Bucksch et al., 2016; Boers et al., 2019; Hygen et al., 2020) and the choice of more appropriate methodological approaches in studies (Rumpf et al., 2019; Colder Carras et al., 2020; Orben, 2020).”
Background
Metadata
- Authors
- Colder Carras, M., Shi, J., Hard, G., & Saldanha, I. J.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-10-26
- Volume
- 15
- Issue
- 10
- Pages
- e0240032
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W3093863699
- Cited by
- 43
- References
- 30
- FWCI
- 1.43
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.