CoChat
Sign Up
Cited by 208Misinformation and Its ImpactsComputational and Text Analysis MethodsMedia and Digital Communication

Digital Trace Data Collection for Social Media Effects Research: APIs, Data Donation, and (Screen) Tracking

Open access
Ohme, Jakob, Araujo, Theo, Boeschoten, Laura, Freelon, Deen, Ram, Nilam, Reeves, Byron B., Robinson, Thomas N.·2024·Communication Methods and Measures
DOI 10.1080/19312458.2023.2181319
Open this paper in CoChat — free accountOpen PDF

Abstract

In social media effects research, the role of specific social media content is understudied, in part attributable to the fact that communication science previously lacked methods to access social media content directly. Digital trace data (DTD) can shed light on textual and audio-visual content of social media use and enable the analysis of content usage on a granular individual level that has been previously unavailable. However, because digital trace data are not specifically designed for research purposes, collection and analysis present several uncertainties. This article is a collaborative effort by scholars to provide an overview of how three methods of digital trace data collection - APIs, data donations, and tracking - can be used in studying the effects of social media content in three important topic areas of communication research: misinformation, algorithmic bias, and well-being. We address the question of how to collect raw social media content data and arrive at meaningful measures with multiple state-of-the-art data collection techniques that can be used to study the effects of social media use on different levels of detail. We conclude with a discussion of best practices for the implementation of each technique, and a comparison of their advantages and disadvantages.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/19312458.2023.2181319

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 65

  1. Abul-Fottouh, D., Song, M. Y., & Gruzd, A. (2020). Examining algorithmic biases in YouTube's recommendations of vaccine videos. International Journal of Medical Informatics, 140, 104175. https://doi.org/10.1016/j.ijmedinf.2020. 104175 · DOI
  2. Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020). Evaluating the fake news problem at the scale of the information ecosystem. Science Advances, 6(14), eaay3539. https://doi.org/10.1126/sciadv.aay3539 · DOI · Paper page
  3. Amaya, A., Biemer, P. P., & Kinyon, D. (2020). Total error in a big data world: Adapting the TSE framework to big data. Journal of Survey Statistics and Methodology, 8(1), 89-119. https://doi.org/10.1093/jssam/smz056 · DOI
  4. Araujo, T., Ausloos, J., van Atteveldt, W., Loecherbach, F., Moeller, J., Ohme, J., Trilling, D., van de Velde, B., de Vreese, C., & Welbers, K. (2022). OSD2F: An open-source data donation framework. Computational Communication Research, 4(2), 372-387. https://doi.org/10.5117/CCR2022.2.001. · DOI
  5. ARAU · DOI
  6. Ausloos, J., & Veale, M. (2021). Researching with data rights. Technology and Regulation, 136-157. http://dx.doi.org/10. 2139/ssrn.3465680 · DOI
  7. Bartley, N., Abeliuk, A., Ferrara, E., & Lerman, K. (2021). Auditing algorithmic bias on Twitter. 13th ACM Web Science Conference 2021, 65-73. https://doi.org/10.1145/34475353462491 . · DOI
  8. Baumgartner, S. E., Sumter, S. R., Petkevič, V., & Wiradhany, W. (2022). A novel iOS data donation approach: automatic processing, compliance, and reactivity in a longitudinal study. Social Science Computer Review, 08944393211071068. https://doi.org/10.1177/08944393211071068 · DOI
  9. Bennett, W. L., & Iyengar, S. (2008). A new era of minimal effects? The changing foundations of political communica- tion. Journal of Communication, 58(4), 707-731.
  10. Boeschoten, L., Ausloos, J., Moeller, J., Araujo, T., & Oberski, D. L. (2020). Digital trace data collection through data donation. ArXiv:2011.09851 [Cs, Stat]. http://arxiv.org/abs/2011.09851 · arXiv
  11. Boeschoten, L., Ausloos, J., Möller, J. E., Araujo, T., & Oberski, D. L. (2022). A framework for privacy preserving digital trace data collection through data donation. Computational Communication Research, 4(2), 388-423. https://doi.org/ 10.5117/CCR2022.2.002. · DOI
  12. BOES Boeschoten, L., Mendrik, A., van der Veen, E., Vloothuis, J., Hu, H., Voorvaart, R., & Oberski, D. L. (2022). Privacy- preserving local analysis of digital trace data: A proof-of-concept. Patterns, 3(3), 100444. https://doi.org/10.1016/j. patter.2022.100444 · DOI
  13. Borges Do Nascimento, I. J., Beatriz Pizarro, A., Almeida, J., Azzopardi-Muscat, N., André Gonçalves, M., Björklund, M., & Novillo-Ortiz, D. (2022). Infodemics and health misinformation: A systematic review of reviews. Bulletin of the World Health Organization, 100(9), 544-561. https://doi.org/10.2471/BLT.21.287654 · DOI
  14. Breuer, J., Kmetty, Z., Haim, M., & Stier, S. (2022). User-centric approaches for collecting Facebook data in the 'post-API age': Experiences from two studies and recommendations for future research. Information, Communication & Society. https://doi.org/10.1080/1369118X.2022.2097015 · DOI
  15. Burgess, J., Angus, D., Carah, N., Andrejevic, M., Hawker, K., Lewis, K., Obeid, A. K., Smith, A., Tan, J., Fordyce, R., Trott, V., & Li, L. 2021. Critical simulation as hybrid digital method for exploring the data operations and vernacular cultures of visual social media platforms. Preprint SocArXiv. https://doi.org/10.31235/osf.io/2cwsu · DOI
  16. Chiatti, A., Cho, M. J., Gagneja, A., Yang, X., Brinberg, M., Roehrick, K., Choudhury, S. R., Ram, N., Reeves, B., & Giles, C. L. (2018). Text extraction and retrieval from smartphone screenshots: Building a repository for life in media. Proceedings of the 33rd Annual ACM Symposium on Applied Computing, 948-955. https://doi.org/10.1145/3167132. 3167236 · DOI
  17. Christner, C., Urman, A., Adam, S., & Maier, M. (2022). Automated tracking approaches for studying online media use: A critical review and recommendations. Communication Methods and Measures, 16(2), 79-95. https://doi.org/10. 1080/19312458.2021.1907841 · DOI
  18. Cohn, J. (2019). The burden of choice: Recommendations, subversion, and algorithmic culture. Rutgers University Press.
  19. Cramer, H., Garcia-Gathright, J., Springer, A., & Reddy, S. (2018). Assessing and addressing algorithmic bias in practice. Interactions, 25(6), 58-63. https://doi.org/10.1145/3278156 · DOI
  20. Cronin, J., von Hohenberg, B. C., Gonçalves, J. F. F., Menchen-Trevino, E., & Wojcieszak, M. (2022). The (null) over- time effects of exposure to local news websites: Evidence from trace data. Journal of Information Technology & Politics, 1-15. https://doi.org/10.1080/19331681.2022.2123878 · DOI
  21. De Vreese, C. H., Boukes, M., Schuck, A., Vliegenthart, R., Bos, L., & Lelkes, Y. (2017). Linking survey and media content data: Opportunities, considerations, and pitfalls. Communication Methods and Measures, 11(4), 221-244. https://doi. org/10.1080/19312458.2017.1380175 · DOI
  22. Diener, E. (1984). Subjective well-being. Psychological Bulletin, 95(3), 542-575. https://doi.org/10.1037/0033-2909.95.3. 542 · DOI · Paper page
  23. Fan, Y., Lehmann, S., & Blok, A. (2022). Extracting the interdisciplinary specialty structures in social media data-based research: A clustering-based network approach. Journal of Informetrics, 16(3), 101310. https://doi.org/10.1016/j.joi. 2022.101310 · DOI
  24. Gaisbauer, F., Pournaki, A., Banisch, S., Olbrich, E., & Guidi, B. (2021). Ideological differences in engagement in public debate on Twitter. Plos One, 16(3), e0249241. https://doi.org/10.1371/journal.pone.0249241 · DOI
  25. Gillespie, T. (2014). The relevance of algorithms. Media Technologies: Essays on Communication, Materiality, and Society, 167(2014), 167.
  26. U.S. presidential election. Science, 363(6425), 374-378. https://doi.org/10.1126/science.aau2706 · DOI · Paper page
  27. Guess, A., Nagler, J., & Tucker, J. (2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science Advances, 5(1), eaau4586. https://doi.org/10.1126/sciadv. · DOI · Paper page
  28. aau4586 · DOI · Paper page
  29. Guo, J. (2022). Deep learning approach to text analysis for human emotion detection from big data. Journal of Intelligent Systems, 31(1), 113-126. https://doi.org/10.1515/jisys-2022-0001 · DOI
  30. Halavais, A. (2019). Overcoming terms of service: A proposal for ethical distributed research. Information, Communication & Society, 22(11), 1567-1581. https://doi.org/10.1080/1369118X.2019.1627386 · DOI
  31. Hooker, S. (2021). Moving beyond "algorithmic bias is a data problem. Patterns, 2(4), 100241. https://doi.org/10.1016/j. patter.2021.100241 · DOI · Paper page
  32. Howison, J., Wiggins, A., & Crowston, K. (2011). Validity issues in the use of social network analysis with digital trace data. Journal of the Association for Information Systems, 12(12), 767-797. https://doi.org/10.17705/1jais.00282 · DOI · Paper page
  33. Kmetty, Z., & Németh, R. (2022). Which is your favorite music genre? A validity comparison of Facebook data and survey data. Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique, 154(1), 82-104. https://doi. org/10.1177/07591063211061754 · DOI
  34. Lazer, D., Hargittai, E., Freelon, D., Gonzalez-Bailon, S., Munger, K., Ognyanova, K., & Radford, J. (2021). Meaningful measures of human society in the twenty-first century. Nature, 595(7866), 189-196. https://doi.org/10.1038/s41586- 021-03660-7 · DOI
  35. Lee, J. H. S., Li, T., Hsu, W., & Lee, M. L. (2021). Repurpose image identification for fake news detection. In C. Strauss, G. Kotsis, A. M. Tjoa, & I. Khalil (Eds.), Database and Expert Systems Applications: 32nd International Conference, DEXA 2021, Virtual Event, September 27-30, 2021, Proceedings, Part II (Vol. 12924). Springer International Publishing. https://doi.org/10.1007/978-3-030-86475-0 · DOI
  36. Lee, J., Reeves, N., Ram, B., & Hamilton, J. (2022). The psychology of poverty and life online: Natural Experiments on the Effects of Smartphone Payday Loan Ads on Psychological Stress. Information, Communication & Society, 1-22. https://doi.org/10.1080/1369118X.2022.2109982 · DOI
  37. Luiten, A., Hox, J. J. C. M., & De Leeuw, E. D. (2020). Survey nonresponse trends and fieldwork effort in the 21st century: Results of an international study across countries and surveys. Journal of Official Statistics, 36(3), 469-487. https://doi. org/10.2478/jos-2020-0025 · DOI
  38. Mackey, T. K., Purushothaman, V., Haupt, M., Nali, M. C., & Li, J. (2021). Application of unsupervised machine learning to identify and characterise hydroxychloroquine misinformation on Twitter. Lancet Digital Health, 3(2), e72-75. https://doi.org/10.1016/S2589-7500(20)30318-6 · DOI
  39. Mellon, J., & Prosser, C. (2017). Twitter and Facebook are not representative of the general population: Political attitudes and demographics of British social media users. Research & Politics, 4(3), 2053168017720008. https://doi.org/10.1177/ 2053168017720008 · DOI
  40. Menchen-Trevino, E. (2016). Web historian: Enabling multi-method and independent research with real-world web browsing history data. IConference 2016 Proceedings (iSchools). https://doi.org/10.9776/16611 . · DOI
  41. Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. NYU Press.
  42. Ohme, J., & Araujo, T. (2022). Digital data donations: A quest for best practices. Patterns, 3(4), 100467. https://doi.org/ 10.1016/j.patter.2022.100467 · DOI
  43. Ohme, J., Araujo, T., de Vreese, C. H., & Piotrowski, J. T. (2021). Mobile data donations: Assessing self-report accuracy and sample biases with the iOS screen time function. Mobile Media & Communication, 9(2), 293-313. https://doi.org/ 10.1177/2050157920959106 · DOI
  44. Orben, A. (2020). Teenagers, screens and social media: A narrative review of reviews and key studies. Social Psychiatry and Psychiatric Epidemiology, 55(4), 407-414. https://doi.org/10.1007/s00127-019-01825-4 · DOI · Paper page
  45. Otto, L. P., Thomas, F., Glogger, I., & De Vreese, C. H. (2022). Linking media content and survey data in a dynamic and digital media environment -mobile longitudinal linkage analysis. Digital Journalism, 10(1), 200-215. https://doi.org/ 10.1080/21670811.2021.1890169 · DOI
  46. Reeves, B., Ram, N., Robinson, T. N., Cummings, J. J., Giles, C. L., Pan, J., Chiatti, A., Cho, M., Roehrick, K., Yang, X., Gagneja, A., Brinberg, M., Muise, D., Lu, Y., Luo, M., Fitzgerald, A., & Yeykelis, L. (2021). Screenomics: A framework to capture and analyze personal life experiences and the ways that technology shapes them. Human-Computer Interaction, 36(2), 150-201. https://doi.org/10.1080/07370024.2019.1578652 · DOI
  47. Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C. (2018). Auditing partisan audience bias within google search. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1-22. https://doi.org/10. 1145/3274417 · DOI · Paper page
  48. Singh, B., & Sharma, D. K. (2022). Predicting image credibility in fake news over social media using multi-modal approach. Neural Computing & Applications, 34(24), 21503-21517. https://doi.org/10.1007/s00521-021-06086-4 · DOI
  49. Stadel, M., & Stulp, G. (2022). Balancing bias and burden in personal network studies. Social Networks, 70, 16-24. https://doi.org/10.1016/j.socnet.2021.10.007 · DOI
  50. Stier, S., Breuer, J., Siegers, P., & Thorson, K. (2020). Integrating survey data and digital trace data: Key issues in developing an emerging field. Social Science Computer Review, 38(5), 503-516. https://doi.org/10.1177/ 0894439319843669 · DOI
  51. Sun, X., Ram, N., Reeves, B., Cho, M. -J., Fitzgerald, A., & Robinson, T. N. (2022). Connectedness and independence of young adults and parents in the digital world: Observing smartphone interactions at multiple timescales using screenomics. Journal of Social and Personal Relationships. https://doi.org/10.1177/02654075221104268 · DOI
  52. Thorson, K., Cotter, K., Medeiros, M., & Pak, C. (2021). Algorithmic inference, political interest, and exposure to news and politics on Facebook. Information, Communication & Society, 24(2), 183-200. https://doi.org/10.1080/1369118X. 2019.1642934 · DOI
  53. Toth, R., & Trifonova, T. (2021). Somebody's watching me: Smartphone use tracking and reactivity. Computers in Human Behavior Reports, 4, 100142. https://doi.org/10.1016/j.chbr.2021.100142 · DOI
  54. Tufekci, Z. (2014). Engineering the public: Big data, surveillance and computational politics. First Monday, https://doi. org/10.5210/fm.v19i7.4901 · DOI
  55. Valkenburg, P. M. (2022). Theoretical foundations of social media uses and effects. In J. Nesi, E. H. Telzer, & M. J. Prinstein (Eds.), Handbook of adolescent digital media use and mental health (1st ed, pp. 39-60). Cambridge University Press. https://doi.org/10.1017/9781108976237.004 · DOI
  56. Valkenburg, P. M., Beyens, I., Meier, A., & Vanden Abeele, M. M. P. (2022). Advancing our understanding of the associations between social media use and well-being. Current Opinion in Psychology, 47, 101357. https://doi.org/10. 1016/j.copsyc.2022.101357 · DOI
  57. Vanden Abeele, M. M. P. (2020). Digital wellbeing as a dynamic construct. Communication Theory. https://doi.org/10. 1093/ct/qtaa024 · DOI · Paper page
  58. van Driel, I. I., Giachanou, A., Pouwels, J. L., Boeschoten, L., Beyens, I., & Valkenburg, P. M. (2022). Promises and pitfalls of social media data donations. Communication Methods and Measures, 16(4), 266-282. https://doi.org/10.1080/ 19312458.2022.2109608 · DOI
  59. Vuorre, M., & Przybylski, A. K. 2022. Global well-being and mental health in the internet age. Preprint PsyArXiv. https:// doi.org/10.31234/osf.io/9tbjy · DOI
  60. Wagner, C., Strohmaier, M., Olteanu, A., Kıcıman, E., Contractor, N., & Eliassi-Rad, T. (2021). Measuring algorithmi- cally infused societies. Nature, 595(7866), 197-204. https://doi.org/10.1038/s41586-021-03666-1 · DOI · Paper page
  61. Wojcieszak, M., Menchen-Trevino, E., Goncalves, J. F. F., & Weeks, B. (2022). Avenues to news and diverse news exposure online: Comparing direct navigation, social media, news aggregators, search queries, and article hyperlinks. The International Journal of Press/politics, 27(4), 194016122110091. https://doi.org/10.1177/19401612211009160 · DOI
  62. Yang, K. -C., Pierri, F., Hui, P. -M., Axelrod, D., Torres-Lugo, C., Bryden, J., & Menczer, F. (2021). The COVID-19 infodemic: Twitter versus Facebook. Big Data & Society, 8(1), 20539517211013860. https://doi.org/10.1177/ 20539517211013861 · DOI
  63. Yang, X., Ram, N., Robinson, T., & Reeves, B. (2019). Using screenshots to predict task switching on smartphones. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1-6. https://doi.org/10. 1145/3290607.3313089 · DOI
  64. Yee, A. Z. H., Yu, R., Lim, S. S., Lim, K. H., Dinh, T. T. A., Loh, L., Hadianto, A., & Quizon, M. (2022). ScreenLife capture: An open-source and user-friendly framework for collecting screenome data from Android smartphones. Behavior Research Methods, 1-18. https://doi.org/10.3758/s13428-022-02006-z · DOI
  65. Zhang, L. C. (2012). Topics of statistical theory for register-based statistics and data integration. Statistica Neerlandica, 66(1), 41-63. https://doi.org/10.1111/j.1467-9574.2011.00508. · DOI

Cited by

Sorted by citations

Authors
Ohme, J., Araujo, T., Boeschoten, L., Freelon, D., Ram, N., Reeves, B. B., & Robinson, T. N.
Affiliations
  • Weizenbaum Institute (DE)
  • University of Amsterdam (NL)
  • Utrecht University (NL)
  • University of North Carolina at Chapel Hill (US)
  • Department of Communication, Stanford University, Stanford, USA
  • Stanford University (US)
  • Stanford Medicine (US)
Corresponding author
Ohme, Jakob
ORCID
Venue
Communication Methods and Measures
Publisher
Informa UK Limited
Published
2023-02-27
Volume
18
Issue
2
Pages
124-141
ISSN
  • 1931-2458 (ISSN-L)
  • 1931-2466
DOI
10.1080/19312458.2023.2181319
OpenAlex
W4322500410
Cited by
208
References
11
Citations by year
2026: 83 · 2025: 69 · 2024: 45 · 2023: 11
FWCI
71.52
Type
journal article
Language
en
Open access
Licence
Locations
  • Communication Methods and Measures (publishedVersion, cc-by-nc-nd)
  • UvA-DARE (University of Amsterdam) (submittedVersion, cc-by-nc-nd)
  • Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by-nc-nd)
  • Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
  • Universitätsbibliothek der FU Berlin Hochschulschriftenstelle u. Dokumentenserver
Funders
Nederlandse Organisatie voor Wetenschappelijk Onderzoek (VI.Vidi.195.152)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Physical Sciences › Computer Science › Computer Networks and Communications › Media and Digital Communication
Keywords
digital traces, social media influence, social media APIs, data donation, social media content, misinformation, algorithmic bias, well-being, data collection methods, research ethics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review