Scales for measuring user engagement with social network sites: A systematic review of psychometric properties
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
In the past decade, various scales have been developed for measuring engagement with social network sites (SNS), but validity concerns have recently been raised about some of them. The present study thus provides a systematic review of the psychometric properties of these scales. This review included articles that aimed at either developing an SNS engagement scale or providing a systematic test of the psychometric properties of the scale. We conducted keyword-based searches of several broad multidisciplinary databases, along with reference list searches and article citation searches. These search strategies yielded a total of 14 reports, revealing validation evidence for 12 SNS engagement scales among 13,861 participants from 11 countries. There was mixed evidence for the various types of validity tests, with some scales having been validated more rigorously with multiple studies and samples while others having not yet been systematically validated. Sampling and acquiescence biases were also present for some scales. The present review provides recommendations for researchers intending to study SNS engagement. Although the literature search was multi-faceted, it may conceivably have missed studies that provided less rigorous validity evidence. Overall, this study contributes to evaluating and strengthening the methodological foundations of SNS research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Social network sites: definition, history, and scholarship
boyd, D. M., & Ellison, N. B. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00393.x · Cited by 16,251
Costello, A. B., & Osborne, J. (2005) · University of Massachusetts Amherst · doi:10.7275/jyj1-4868 · Cited by 10,297
Ellison, N. B., Steinfield, C., & Lampe, C. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00367.x · Cited by 9,939
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., & Erikson, P. (2005) · Value in Health · doi:10.1111/j.1524-4733.2005.04054.x · Cited by 5,521
A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires
Hinkin, T. R. (1998) · Organizational Research Methods · doi:10.1177/109442819800100106 · Cited by 4,095
Scale Development Research: A Content Analysis and Recommendations for Best Practices
Worthington, R. L., & Whittaker, T. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288127 · Cited by 4,087
Starting at the beginning: an introduction to coefficient alpha and internal consistency
Streiner, D. L. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8001_18 · Cited by 3,794
Conventional wisdom on measurement: A structural equation perspective.
Bollen, K., & Lennox, R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.305 · Cited by 3,618
Yin, R. K. (2009). Case study research: Design and methods (4th Ed.). Thousand Oaks, CA: Sage.
Aberdeen, T. (2013) · The Canadian Journal of Action Research · doi:10.33524/cjar.v14i1.73 · Cited by 3,250
Social capital, self-esteem, and use of online social network sites: A longitudinal analysis.
Steinfield, C., Ellison, N. B., & Lampe, C. (2008) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2008.07.002 · Cited by 1,919
Development of a Facebook Addiction Scale
Andreassen, C. S., Torsheim, T., Brunborg, G. S., & Pallesen, S. (2012) · Psychological Reports · doi:10.2466/02.09.18.pr0.110.2.501-517 · Cited by 1,884
Personality and motivations associated with Facebook use
Ross, C., Orr, E. S., Sisic, M., Arseneault, J. M., Simmering, M. G., & Orr, R. R. (2009) · Computers in Human Behavior · doi:10.1016/j.chb.2008.12.024 · Cited by 1,728
Research Synthesis and Meta-Analysis: A Step-by-Step Approach
Cooper, H. (2017) · doi:10.4135/9781071878644 · Cited by 1,716
What is user engagement? A conceptual framework for defining user engagement with technology
O'Brien, H. L., & Toms, E. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20801 · Cited by 1,397
Rosenberg, M. S. (2005) · Evolution · doi:10.1111/j.0014-3820.2005.tb01004.x · Cited by 1,285 · cited 1×
“Publication bias is commonly recognized as a potential threat to the comprehensiveness of systematic reviews (see e.g., Bradley & Gupta, 1997; Rosenberg, 2005).”
BackgroundDo social network sites enhance or undermine subjective well-being? A critical review
Verduyn, P., Ybarra, O., Résibois, M., Jonides, J., & Kross, E. (2017) · Social Issues and Policy Review · doi:10.1111/sipr.12033 · Cited by 1,209 · cited 1×
“…as euphoria and cognitive salience (Charlton & Danforth, 2007), a robust body of theoretical and empirical work has shown that they are distinct constructs, particularly their relationships with different indicators of psychological well-being (e.g., Lin, Hung, Fang, & Tu, 2015; Wan & Chiou, 2006).”
BackgroundInternet Gaming Addiction: A Systematic Review of Empirical Research
Kuss, D. J., & Griffiths, M. D. (2012) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-011-9318-5 · Cited by 1,113
A tale of two sites: Twitter vs. Facebook and the personality predictors of social media usage
Hughes, D. J., Rowe, M., Batey, M., & Lee, A. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2011.11.001 · Cited by 1,011
Distinguishing addiction and high engagement in the context of online game playing
Charlton, J. P., & Danforth, I. D. W. (2007) · Computers in Human Behavior · doi:10.1016/j.chb.2005.07.002 · Cited by 946
Beyens, I., Frison, E., & Eggermont, S. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.05.083 · Cited by 847
The Media and Technology Usage and Attitudes Scale: An empirical investigation
Rosen, L. D., Whaling, K., Carrier, L. M., Cheever, N. A., & Rokkum, J. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.06.006 · Cited by 788
Comparing alternatives to the web of science for coverage of the social sciences’ literature
Norris, M., & Oppenheim, C. (2007) · Journal of Informetrics · doi:10.1016/j.joi.2006.12.001 · Cited by 733
Cheng, C., & Li, A. Y.-L. (2014) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2014.0317 · Cited by 727
Self-citationThe uses and abuses of Facebook: A review of Facebook addiction
Ryan, T., Chester, A., Reece, J., & Xenos, S. (2014) · Journal of Behavioral Addictions · doi:10.1556/jba.3.2014.016 · Cited by 714
Social networking on smartphones: When mobile phones become addictive
Salehan, M., & Negahban, A. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.07.003 · Cited by 700
The benefits and dangers of enjoyment with social networking websites
Turel, O., & Serenko, A. (2012) · European Journal of Information Systems · doi:10.1057/ejis.2012.1 · Cited by 684
The Accuracy of Self-Reported Internet Use—A Validation Study Using Client Log Data
Scharkow, M. (2016) · Communication Methods and Measures · doi:10.1080/19312458.2015.1118446 · Cited by 408 · cited 5×
“The rise of the Internet has spurred interest across multiple social sciences disciplines in how this new technology can transform social dynamics (e.g., Kraut et al., 2002; Wellman, 2002).”
BackgroundInfluentialCheng, C., Cheung, S. F., Chio, J. H.-M., & Chan, M.-P. S. (2013) · Psychological Bulletin · doi:10.1037/a0028596 · Cited by 407
Self-citation“What do they snapchat about?” Patterns of use in time-limited instant messaging service
Piwek, L., & Joinson, A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2015.08.026 · Cited by 281
Development and validation of a social media use integration scale.
Jenkins-Guarnieri, M. A., Wright, S. L., & Johnson, B. (2013) · Psychology of Popular Media Culture · doi:10.1037/a0030277 · Cited by 273 · cited 1×
“…shown that the FBI scale is reliable (i.e., Cronbach's a> .70), including instances where it was adapted for use in diverse contexts such as SNS engagement in general (Salehan & Negahban, 2013), Snapchat (Piwek & Joinson, 2016), and cross-cultural Facebook interaction (Jiang & Bruijn, 2013).”
MethodUsing APIs for Data Collection on Social Media
Lomborg, S., & Bechmann, A. (2014) · The Information Society · doi:10.1080/01972243.2014.915276 · Cited by 258
A meta-analysis of social networking online and social capital
Liu, D., Ainsworth, S. E., & Baumeister, R. F. (2016) · Review of General Psychology · doi:10.1037/gpr0000091 · Cited by 205 · cited 2×
“A recent meta-analysis by Liu, Ainsworth, and Baumeister (2016) found that SNS “intensity” (the most commonly used conceptualization of SNS engagement) had positive associations with both bridging and bonding social capital (r's¼ .35 and .27, respectively).”
BackgroundMethodAlhabash, S., Chiang, Y.-H., & Huang, K. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.03.033 · Cited by 112
Gravitating towards Facebook (GoToFB): What it is? and How can it be measured?
Aladwani, A. M. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.01.005 · Cited by 65 · cited 1×
“, 2016), the Middle East (i.e., Kuwait; Aladwani, 2014), and North America (e.”
Background
Cited by
Short, C. E., DeSmet, A., Woods, C., Williams, S. L., Maher, C., Middelweerd, A., Müller, A. M., Wark, P. A., Vandelanotte, C., Poppe, L., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9397 · Cited by 400
“Lessons may be gleaned from the scales developed to assess social networking intensity [50].”
BackgroundGrassini, S. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1191628 · Cited by 330
Tandon, A., Dhir, A., Almugren, I., AlNemer, G. N., & Mäntymäki, M. (2021) · Internet Research · doi:10.1108/intr-11-2019-0455 · Cited by 315
Consumer engagement in social media brand communities: A literature review
Santos, Z. R., Cheung, C. M. K., Coelho, P. S., & Rita, P. (2022) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102457 · Cited by 265
Hawes, T., Zimmer-Gembeck, M. J., & Campbell, S. M. (2020) · Body Image · doi:10.1016/j.bodyim.2020.02.010 · Cited by 186
Metadata
- Authors
- Sigerson, L., & Cheng, C.
- Venue
- Computers in Human Behavior
- Publisher
- Elsevier BV
- Published
- 2018-06
- Volume
- 83
- Pages
- 87-105
- ISSN
- 0747-5632 (ISSN-L)
- OpenAlex
- W2791971316
- Cited by
- 142
- References
- 36
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.