Validity Issues in the Use of Social Network Analysis with Digital Trace Data
Abstract
There is an exciting natural match between social network analysis methods and the growth of data sources produced by social interactions via information technologies, from online communities to corporate information systems. Information Systems researchers have not been slow to embrace this combination of method and data. Such systems increasingly provide “digital trace data” that provide new research opportunities. Yet digital trace data are substantively different from the survey and interview data for which network analysis measures and interpretations were originally developed. This paper examines 10 validity issues associated with the combination of digital trace data and social network analysis methods, with examples from the IS literature, to provide recommendations for improving the validity of future research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Borgatti, S. P. (2005) · Social Networks · doi:10.1016/j.socnet.2004.11.008 · Cited by 3,191 · see citing works
Graphs over time: densification laws, shrinking diameters and possible explanations
Leskovec, J., Kleinberg, J., & Faloutsos, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081893 · Cited by 2,281 · see citing works
Social Networks and Organizations
Kilduff, M., & Tsai, W. (2003) · doi:10.4135/9781849209915 · Cited by 1,581 · see citing works
Some antecedents and effects of trust in virtual communities
Ridings, C. M., Gefen, D., & Arinze, B. (2002) · The Journal of Strategic Information Systems · doi:10.1016/s0963-8687(02)00021-5 · Cited by 1,423 · cited 1× · see citing works
“There is an exciting natural match between Social Network Analysis (SNA) and the growing phenomenon of social interaction through digital platforms and technologies, from online communities to corporate information systems (Agarwal, Gupta, & Kraut, 2008).”
Casting the Net: A Multimodal Network Perspective on User–System Interactions
Kane, G. C., & Alavi, M. (2008) · Information Systems Research · doi:10.1287/isre.1070.0158 · Cited by 135 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · see citing works
“It also includes temporal validity , which captures to what extent constructs change over time (Howison et al., 2011) and may invalidate previous conclusions about societal and/or platform phenomena; e.g., see the case of Google Flu Trends (Lazer et al., 2014).”
BackgroundInfluentialMethods in predictive techniques for mental health status on social media: a critical review
Chancellor, S., & De Choudhury, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0233-7 · Cited by 557 · see citing works
Lehrer, C., Wieneke, A., vom Brocke, J., Jung, R., & Seidel, S. (2018) · Journal of Management Information Systems · doi:10.1080/07421222.2018.1451953 · Cited by 404 · see citing works
Tampering with Twitter’s Sample API
Pfeffer, J., Mayer, K., & Morstatter, F. (2018) · EPJ Data Science · doi:10.1140/epjds/s13688-018-0178-0 · Cited by 135 · see citing works
On the interpretation of digital trace data in communication and social computing research
Freelon, D. (2014) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2013.875018 · Cited by 132 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763 · see citing works
- Authors
- University of Texas at Austin, Howison, J., Wiggins, A., Syracuse University, Crowston, K., & Syracuse University
- Affiliations
- The University of Texas at Austin (US)
- Syracuse University (US)
- Venue
- Journal of the Association for Information Systems
- Publisher
- Association for Information Systems
- Published
- 2011-12
- Volume
- 12
- Issue
- 12
- Pages
- 767-797
- ISSN
- 1536-9323 (ISSN-L)
- OpenAlex
- W2121712502
- Cited by
- 369
- References
- 5
- Citations by year
- 2026: 28 · 2025: 18 · 2024: 27 · 2023: 29 · 2022: 26 · 2021: 38 · 2020: 37 · 2019: 32 · 2018: 23 · 2017: 20 · 2016: 20 · 2015: 16 · 2014: 27 · 2013: 16 · 2012: 8
- FWCI
- 13.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the Association for Information Systems (publishedVersion)
- Journal of the Association for Information Systems (submittedVersion)
- Topics
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Social Sciences › Communication › Knowledge Management and Sharing
- Keywords
- social network analysis, digital traces, validity, online communities, corporate information systems, social interaction data, research methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
