Big questions for social media big data: Representativeness, validity and other methodological pitfalls
Open accessAbstract
Large-scale databases of human activity in social media have captured scientific and policy attention, producing a flood of research and discussion. This paper considers methodological and conceptual challenges for this emergent field, with special attention to the validity and representativeness of social media big data analyses. Persistent issues include the over-emphasis of a single platform, Twitter, sampling biases arising from selection by hashtags, and vague and unrepresentative sampling frames. The socio-cultural complexity of user behavior aimed at algorithmic invisibility (such as subtweeting, mock-retweeting, use of “screen captures” for text, etc.) further complicate interpretation of big data social media. Other challenges include accounting for field effects, i.e. broadly consequential events that do not diffuse only through the network under study but affect the whole society. The application of network methods from other fields to the study of human social activity may not always be appropriate. The paper concludes with a call to action on practical steps to improve our analytic capacity in this promising, rapidly-growing field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Social data opens unprecedented opportunities to answer significant questions about society, policies, and health.
“Social data opens unprecedented opportunities to answer significant questions about society, policies, and health, being recognized as one core reason behind progress in many areas of computing (e.g., crisis informatics, digital health, computational social science) (Crawford and Finn, 2014; Tufekci, 2014; Yom-Tov, 2016).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
The Introduction (page unavailable) says social media big data enables insights into human behavior and observation of social phenomena at a previously unthinkable scale; the Abstract also notes scientific and policy attention. These passages support the broad claim of new opportunities, but they do not establish the specific claim about answering health questions or being a core reason for progress in the listed computing fields. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe behaviors observed in online social platforms are partially determined by platform capabilities, which are engineered toward certain goals.
“The behaviors we observe in online social platforms are partially determined by platform capabilities, which are engineered toward certain goals”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
In “Model Organisms and Research,” the paper says platform affordances help structure behavior and gives examples of how Twitter’s features shape interactions. It does not say those capabilities are engineered toward particular goals, so that part of the claim is unsupported by our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe functional peculiarities of social platforms may introduce population and behavioral biases by influencing which user demographics are drawn to each platform and the kind of actions they are more likely to perform.
“The functional peculiarities of social platforms may introduce population (section 3.2) and behavioral biases (section 3.3) by influencing which user demographics are drawn to each platform and the kind of actions they are more likely to perform”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
In “Model Organisms and Research” (page unavailable), the paper says platform affordances structure behavior and that platform users differ by demographic factors, raising representativeness concerns. It also links Twitter’s lightweight interface to its use in particular circumstances. It does not establish that platform functions influence which demographic groups are drawn to each platform, so that causal part of the claim is not supported. (LLM, conf=0.84)
Experimental — accuracy not yet measuredOverlooking the impact of norms can impact any social data analysis studying or making assumptions about user behavior.
“Overlooking the impact of norms can impact any social data analysis studying or making assumptions about user behavior”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
The section “Methodological Considerations 1. Model Organisms and Research” says Twitter’s retweet mechanism generates status-related behaviors and norms that may not translate to other platforms. “What’s in a Retweet? Understanding our Data” shows how assumptions about retweets can misinterpret user behavior. These passages support the concern for social-media analysis, but not the claim’s extension to any social data analysis. (LLM, conf=0.82)
Experimental — accuracy not yet measuredIf data is organized along a certain attribute, and there are multiple independent factors that result in the attribute taking a certain value, analyzing data entries with this value is equivalent to conditioning on it, and may result in spurious patterns of association among these factors.
“if the data is organized along a certain attribute (e.g., the presence of a keyword or hashtag), and there are multiple independent factors that result in the attribute taking a certain value, analyzing data entries with this value is equivalent to conditioning on it, and may result in spurious patterns of association among these factors (Blyth, 1972; Tufekci, 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Such implications are not always well understood or recognized.
“Yet, such implications are not always well understood or recognized (Tufekci, 2014; O'Neil, 2016), as many seem to assume that these data, and the frameworks used to handle them, are adequate, often as-is, for the problem at hand, with little or no scrutiny.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
There are lingering questions about what can be learned from social data about real-world phenomena which have yet to be rigorously addressed.
“Regardless of how large or varied social data are, there are also lingering questions about what can be learned from them about real-world phenomena (online or offline)—which have yet to be rigorously addressed (boyd and Crawford, 2012; Ruths and Pfeffer, 2014; Tufekci, 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
Research agendas tend to be opportunistically driven by access to data, tools, or ease of analysis.
“A key concern is that research agendas tend to be opportunistically driven by access to data, tools, or ease of analysis (Ruths and Pfeffer, 2014; Tufekci, 2014; Weller and Gorman, 2015); or, as Baeza-Yates (2013) puts it, “we see a lot of data mining for the sake of it.””
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
Much research is concentrated on data from a few platforms, most notably Twitter.
“much research is concentrated on data from a few platforms, most notably Twitter.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
Functional biases make conclusions from research studies difficult to generalize or transfer, as each platform exhibits its own structural differences, which can lead to platform-specific phenomena that are often overlooked.
“Functional biases make conclusions from research studies difficult to generalize or transfer, as each platform exhibits its own structural differences (Tufekci, 2014), which can lead to platform-specific phenomena (Ruths and Pfeffer, 2014) that are often overlooked.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
We often know what people write, but not what they read, and we may know who clicked on or "liked" something, but not who read it or watched it.
“For instance, we often know what people write, but not what they read, and we may know who clicked on or “liked” something, but not who read it or watched it”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
The choice of keywords in keyword-based queries shapes the resulting datasets.
“The choice of keywords in keyword-based queries shapes the resulting datasets.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
Different hashtags used in the same context may be associated with distinct social, political or cultural frames, and samples built using them may embed different dimensions of the data.
“different hashtags used in the same context (e.g., during a political event) may be associated with distinct social, political or cultural frames, and, thus, samples built using them may embed different dimensions of the data (Tufekci, 2014)”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
Many datasets are built around dependent variables, deciding to include a user or a piece of content depending on the inclusion of a variable under analysis; this may result in apparent patterns of association that fail to generalize.
“Many datasets are built around dependent variables (see section 5.2), deciding to include a user or a piece of content depending on the inclusion of a variable under analysis (Tufekci, 2014); this may result in apparent patterns of association that fail to generalize.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
The same mechanism or process may capture different signals across contexts, but such distinctions are often unintelligible and hard to make in an automated fashion when looking at a system output or at an analysis results in aggregate.
“The same mechanism or process may capture different signals across contexts (Tufekci, 2014), but such distinctions are often unintelligible and hard to make in an automated fashion when looking at a system output or at an analysis results in aggregate (Rost et al., 2013; Tufekci, 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Tufekci, 2014
If a study investigates the effectiveness of a particular treatment based on data collected from a specific time period, the findings may not reflect the current healthcare landscape or advancements in medical practices.
“For instance, if a study investigates the effectiveness of a particular treatment based on data collected from a specific time period, the findings may not reflect the current healthcare landscape or advancements in medical practices [91].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [91]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
What is Twitter, a social network or a news media?
Kwak, H., Lee, C., Park, H., & Moon, S. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772751 · Cited by 6,652 · see citing works
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479 · see citing works
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365 · see citing works
Measuring user influence in twitter: The million follower fallacy
Cha, M., Haddadi, H., Benevenuto, F., & Gummadi, K. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14033 · Cited by 3,022 · cited 1× · see citing works
“Another important social media study, based on Twitter, speaks of in-degree (number of followers) as a user’s popularity, and retweets as influence (Cha et al., 2010).”
BackgroundSocial Media and the Decision to Participate in Political Protest: Observations From Tahrir Square
Tüfekçi, Z., & Wilson, C. (2012) · Journal of Communication · doi:10.1111/j.1460-2466.2012.01629.x · Cited by 1,540 · see citing works
Whose space? differences among users and non-users of social network sites
Hargittai, E. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00396.x · Cited by 1,286 · see citing works
Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures
Golder, S. A., & Macy, M. W. (2011) · Science · doi:10.1126/science.1202775 · Cited by 1,155 · cited 1× · see citing works
“…and allow us to observe social phenomena at a previously unthinkable level, such as the mood oscillations of millions of people in 84 countries (Golder et al., 2011), or in cases where there is arguably no other feasible method Copyright © 2014, Association for the Advancement of Artificial…”
MethodRomero, D. M., Meeder, B., & Kleinberg, J. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963503 · Cited by 1,082 · see citing works
Information contagion: an empirical study of the spread of news on digg and twitter social networks
Lerman, K., & Ghosh, R. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14021 · Cited by 813 · see citing works
Revisiting the Foundations of Network Analysis
Butts, C. T. (2009) · Science · doi:10.1126/science.1171022 · Cited by 544 · see citing works
Ecological Developmental Biology: Developmental Biology Meets the Real World
Gilbert, S. F. (2001) · Developmental Biology · doi:10.1006/dbio.2001.0210 · Cited by 468 · see citing works
Quantifying the invisible audience in social networks
Bernstein, M. S., Bakshy, E., Burke, M., & Karrer, B. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470658 · Cited by 357 · see citing works
The Arab uprising: the unfinished revolutions of the new Middle East
(2012) · Choice Reviews Online · doi:10.5860/choice.50-1111 · Cited by 325 · see citing works
Model systems in developmental biology
Bolker, J. A. (1995) · BioEssays · doi:10.1002/bies.950170513 · Cited by 293 · see citing works
The choice of model organisms in evo–devo
Jenner, R. A., & Wills, M. A. (2007) · Nature Reviews Genetics · doi:10.1038/nrg2062 · Cited by 183 · see citing works
Whither Model Organism Research?
Fields, S., & Johnston, M. (2005) · Science · doi:10.1126/science.1108872 · Cited by 153 · see citing works
On the interpretation of digital trace data in communication and social computing research
Freelon, D. (2014) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2013.875018 · Cited by 132 · see citing works
The dynamics of health behavior sentiments on a large online social network
Salathé, M., Vu, D. Q., Khandelwal, S., & Hunter, D. R. (2013) · EPJ Data Science · doi:10.1140/epjds16 · Cited by 116 · see citing works
Adar, E., Weld, D. S., Bershad, B. N., & Gribble, S. D. (2007) · International World Wide Web Conference (WWW) · doi:10.1145/1242572.1242595 · Cited by 115 · see citing works
Syria’s socially mediated civil war
Lynch, M., Freelon, D., & Aday, S. (2013) · Cited by 70 · see citing works
Ecological Developmental Biology: Developmental Biology Meets the Real World1
Gilbert, S. F. (2004) · Russian Journal of Developmental Biology · doi:10.1023/b:rudo.0000049607.17338.8a · Cited by 48 · see citing works
Mann, A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1524881113 · Cited by 40 · see citing works
Towards Supporting Search over Trending Events with Social Media
Kairam, S., Morris, M. R., Teevan, J. B., Liebling, D. J., & Dumais, S. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14426 · Cited by 38 · see citing works
Dunning, T., & Freedman, D. A. (2007) · SAGE Publications Ltd eBooks · doi:10.4135/9781848607958.n12 · Cited by 11 · see citing works
LARGE DATA SETS IN PRIMARY CARE RESEARCH
Meiman, J., & Freund, J. E. (2012) · The Annals of Family Medicine · doi:10.1370/afm.1441 · Cited by 6 · see citing works
Flagged first, then by citations
1 of the 27 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [4]4. Allison, P.D. 2001. Missing data. Thousand Oaks, CA: SAGE.
- [8]8. boyd, d., and Crawford, K. 2012. Critical Questions for Big Data. Information, Communication & Society 15 (5): 662-679.
- [28]28. Pray, L. & Zhaurova, K. (2008) Barbara McClintock and the discovery of jumping genes (transposons). Nature Education 1(1):169
Cited by
Sorted by citations
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
Algorithms at Work: The New Contested Terrain of Control
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0174 · Cited by 2,732 · see citing works
Fake news on Twitter during the 2016 U.S. presidential election
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019) · Science · doi:10.1126/science.aau2706 · Cited by 1,803 · see citing works
Xiang, Z., Du, Q., Ma, Y., & Fan, W. (2016) · Tourism Management · doi:10.1016/j.tourman.2016.10.001 · Cited by 1,128 · see citing works
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043 · see citing works
Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7347 · Cited by 881 · see citing works
Social media analytics – Challenges in topic discovery, data collection, and data preparation
Stieglitz, S., Mirbabaie, M., Roß, B., & Neuberger, C. (2017) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2017.12.002 · Cited by 848 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · see citing works
“Results of various methods to collect, measure, or process data should be routinely juxtaposed (Ruths and Pfeffer, 2014; Tufekci, 2014).”
BackgroundMethodInfluentialSocial media for large studies of behavior
Ruths, D., & Pfeffer, J. (2014) · Science · doi:10.1126/science.346.6213.1063 · Cited by 648 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
“temporal bias Bias that arises from differences in populations and behaviors over time [147, 285].”
BackgroundCastillo, D. J., & Sánchez‐Fernández, R. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.07.009 · Cited by 580 · see citing works
Badawy, A., Ferrara, E., & Lerman, K. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508646 · Cited by 353 · see citing works
Social Media– and Internet-Based Disease Surveillance for Public Health
Aiello, A. E., Renson, A., & Zivich, P. N. (2020) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040119-094402 · Cited by 354 · see citing works
Showing They Care (Or Don’t): Affective Publics and Ambivalent Climate Activism on TikTok
Hautea, S., Parks, P., Takahashi, B., & Zeng, J. (2021) · Social Media + Society · doi:10.1177/20563051211012344 · Cited by 351 · see citing works
Camacho, D., Panizo-LLedot, Á., Bello-Orgaz, G., González-Pardo, A., & Cambria, E. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.05.009 · Cited by 339 · see citing works
Social Media data: Challenges, opportunities and limitations in urban studies
Ciriquián, P. M., Serrano-Estrada, L., & Nolasco‐Cirugeda, A. (2018) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2018.11.001 · Cited by 335 · see citing works
SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours
Derczynski, L., Bontcheva, K., Liakata, M., Procter, R., Wong Sak Hoi, G., & Zubiaga, A. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2006 · Cited by 323 · see citing works
Misinformation on Misinformation: Conceptual and Methodological Challenges
Altay, S., Berriche, M., & Acerbi, A. (2023) · Social Media + Society · doi:10.1177/20563051221150412 · Cited by 307 · see citing works
Data ex Machina: Introduction to Big Data
Lazer, D. M. J., & Radford, J. P. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-060116-053457 · Cited by 305 · see citing works
Hate Lingo: A Target-Based Linguistic Analysis of Hate Speech in Social Media
ElSherief, M., Kulkarni, V., Nguyen, D., Yang Wang, W., & Belding, E. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15041 · Cited by 294 · see citing works
Right-Wing YouTube: A Supply and Demand Perspective
Munger, K., & Phillips, J. (2022) · The International Journal of Press/Politics · doi:10.1177/1940161220964767 · Cited by 265 · see citing works
Tweet for Behavior Change: Using Social Media for the Dissemination of Public Health Messages
Gough, A., Hunter, R. F., Ajao, O., Jurek, A., McKeown, G. J., Hong, J. S., Barrett, E., Ferguson, M., McElwee, G., McCarthy, M., et al. (2017) · JMIR Public Health and Surveillance · doi:10.2196/publichealth.6313 · Cited by 262 · see citing works
CREDBANK: A Large-Scale Social Media Corpus With Associated Credibility Annotations
Mitra, T., & Gilbert, É. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14625 · Cited by 236 · see citing works
A methodology for mapping Instagram hashtags
Highfield, T., & Leaver, T. B. (2014) · First Monday · doi:10.5210/fm.v20i1.5563 · Cited by 219 · see citing works
'Too Gay for Facebook': Presenting LGBTQ+ Identity Throughout the Personal Social Media Ecosystem
DeVito, M. A., Walker, A. M., & Birnholtz, J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274313 · Cited by 219 · see citing works
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261 · see citing works
Pouyanfar, S., Yang, Y., Chen, S., Shyu, M., & Iyengar, S. S. (2018) · ACM Computing Surveys · doi:10.1145/3150226 · Cited by 195 · see citing works
Crawford, K., & Finn, M. (2015) · GeoJournal · doi:10.1007/s10708-014-9597-z · Cited by 191 · see citing works
Social Media Analyses for Social Measurement
Schober, M. F., Josh Pasek, B., Guggenheim, L., Lampe, C. A. C., & Conrad, F. G. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfv048 · Cited by 188 · see citing works
Big Data Analytics, Tourism Design and Smart Tourism
Xiang, Z., & Fesenmaier, D. R. (2016) · Tourism on the verge · doi:10.1007/978-3-319-44263-1_17 · Cited by 178 · see citing works
Veletsianos, G., & Kimmons, R. M. (2016) · The Internet and Higher Education · doi:10.1016/j.iheduc.2016.02.002 · Cited by 175 · see citing works
Twitter Usage in IndonesiaPreprint
Carley, K. M., Malik, M. M., Landwehr, P. M., Pfeffer, J., & Kowalchuck, M. J. (2015) · SSRN Electronic Journal · doi:10.1016/j.ssci.2016.04.002 · Cited by 220 · see citing works
Population Bias in Geotagged Tweets
Malik, M. M., Lamba, H., Nakos, C., & Pfeffer, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i4.14688 · Cited by 168 · see citing works
Lee, K. K., & Yu, C. (2018) · Journal of Air Transport Management · doi:10.1016/j.jairtraman.2018.05.004 · Cited by 163 · see citing works
Peer to Peer Hate: Hate Speech Instigators and Their Targets
ElSherief, M., Nilizadeh, S., Nguyen, D., Vigna, G., & Belding, E. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15038 · Cited by 158 · see citing works
(2016) · Thammasat University Digital Collections · Cited by 153 · see citing works
Seabrook, E. M., Kern, M. L., Fulcher, B. D., & Rickard, N. S. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9267 · Cited by 144 · see citing works
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152 · see citing works
Communication Technology and Protest
Little, A. T. (2016) · The Journal of Politics · doi:10.1086/683187 · Cited by 192 · see citing works
How 'social' are destinations? Examining European DMO social media usage
Usakli, A., Koç, B., & Sönmez, S. (2017) · Journal of Destination Marketing & Management · doi:10.1016/j.jdmm.2017.02.001 · Cited by 136 · see citing works
Sarcastic sentiment detection in tweets streamed in real time: a big data approach
Bharti, S. K., Vachha, B., Pradhan, R. K., Babu, K. S., & Jena, S. K. (2016) · Digital Communications and Networks · doi:10.1016/j.dcan.2016.06.002 · Cited by 143 · see citing works
Mapping Controversies with Social Media: The Case for Symmetry
Marres, N., & Moats, D. (2015) · Social Media + Society · doi:10.1177/2056305115604176 · Cited by 137 · see citing works
Promises and Pitfalls of Using Digital Traces for Demographic Research
Cesare, N., Lee, H. E., McCormick, T. H., Spiro, E. S., & Zagheni, E. (2018) · Demography · doi:10.1007/s13524-018-0715-2 · Cited by 137 · see citing works
Tampering with Twitter’s Sample API
Pfeffer, J., Mayer, K., & Morstatter, F. (2018) · EPJ Data Science · doi:10.1140/epjds/s13688-018-0178-0 · Cited by 135 · see citing works
Morris, M., Zolyomi, A., Yao, C., Bahram, S., Bigham, J., & Kane, S. K. (2016) · International Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858116 · Cited by 157 · see citing works
Identifying Tourist Places of Interest Based on Digital Imprints: Towards a Sustainable Smart City
Encalada-Abarca, L., Boavida-Portugal, I., Ferreira, C. C., & Rocha, J. (2017) · Sustainability · doi:10.3390/su9122317 · Cited by 132 · see citing works
The unbearable emptiness of tweeting—About journal articles
Robinson‐García, N., Costas, R., Isett, K. R., Melkers, J. E., & Hicks, D. M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0183551 · Cited by 130 · see citing works
(Why) Is Misinformation a Problem?
Adams, Z., Osman, M., Bechlivanidis, C., & Meder, B. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221141344 · Cited by 130 · see citing works
Ghermandi, A., Langemeyer, J., Berkel, D. V., Calcagni, F., Depietri, Y., Vigl, L. E., Fox, N., Havinga, I., Jäger, H., Kaiser, N. N., et al. (2023) · One Earth · doi:10.1016/j.oneear.2023.02.008 · Cited by 132 · see citing works
From value co-creation to value co-destruction? The case of dockless bike sharing in China
Yin, J., Qian, L., & Shen, J. (2018) · Transportation Research Part D Transport and Environment · doi:10.1016/j.trd.2018.12.004 · Cited by 119 · see citing works
Social Network Analysis: A Survey on Process, Tools, and Application
Singh, S. S., Muhuri, S., Mishra, S., Srivastava, D., Shakya, H. K., & Kumar, N. (2024) · ACM Computing Surveys · doi:10.1145/3648470 · Cited by 119 · see citing works
Schöne, J. P., Parkinson, B., & Goldenberg, A. (2021) · Affective Science · doi:10.1007/s42761-021-00057-7 · Cited by 114 · see citing works
Chaudhary, K., Alam, M., Al-Rakhami, M. S., & Gumaei, A. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00466-2 · Cited by 113 · see citing works
The dynamics of information-driven coordination phenomena: A transfer entropy analysis
Borge‐Holthoefer, J., Perra, N., Gonçalves, B., González-Bailón, S., Arenas, À., Moreno, Y., & Vespignani, A. (2016) · Science Advances · doi:10.1126/sciadv.1501158 · Cited by 110 · see citing works
Contemporary Social Movements in a Hybrid Media Environment
Caren, N., Andrews, K. T., & Lu, T. (2020) · Annual Review of Sociology · doi:10.1146/annurev-soc-121919-054627 · Cited by 108 · see citing works
Characterizing the 2016 Russian IRA influence campaign
Badawy, A., Addawood, A., Lerman, K., & Ferrara, E. (2019) · Social Network Analysis and Mining · doi:10.1007/s13278-019-0578-6 · Cited by 109 · see citing works
Choi, Y. H., & Bazarova, N. N. (2014) · Human Communication Research · doi:10.1111/hcre.12053 · Cited by 108 · see citing works
Brooker, P., Barnett, J. C., & Cribbin, T. F. (2016) · Big Data & Society · doi:10.1177/2053951716658060 · Cited by 106 · see citing works
Protest Activity Detection and Perceived Violence Estimation from Social Media Images
Won, D., STEINERT-THRELKELD, Z., & Joo, J. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123282 · Cited by 111 · see citing works
The social amplification of risk on Twitter: the case of ash dieback disease in the United Kingdom
Fellenor, J., Barnett, J. C., Potter, C., Urquhart, J., Mumford, J. D., & Quine, C. P. (2017) · Journal of Risk Research · doi:10.1080/13669877.2017.1281339 · Cited by 103 · see citing works
Social Set Analysis: A Set Theoretical Approach to Big Data Analytics
Vatrapu, R., Mukkamala, R. R., Hussain, A., & Flesch, B. (2016) · IEEE Access · doi:10.1109/access.2016.2559584 · Cited by 102 · see citing works
Adapting to supply chain 4.0: an explorative study of multinational companies
Makris, D., Hansen, Z. N. L., & Khan, O. (2019) · Supply Chain Forum an International Journal · doi:10.1080/16258312.2019.1577114 · Cited by 101 · see citing works
A computational study of mental health awareness campaigns on social media
Saha, K., Torous, J., Ernala, S. K., Rizuto, C., Stafford, A., & De Choudhury, M. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz028 · Cited by 93 · see citing works
“Hunger Hurts but Starving Works”
Pater, J. A., Haimson, O. L., Andalibi, N., & Mynatt, E. D. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2820030 · Cited by 101 · see citing works
Gaffney, D., & Matias, J. N. (2018) · PLoS ONE · doi:10.1371/journal.pone.0200162 · Cited by 99 · see citing works
How to Study the City on Instagram
Boy, J. D., & Uitermark, J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0158161 · Cited by 98 · see citing works
The Double-Edged Sword of Big Data in Organizational and Management Research
Wenzel, R., & Quaquebeke, N. V. (2017) · Organizational Research Methods · doi:10.1177/1094428117718627 · Cited by 104 · see citing works
Predictive Analysis on Twitter: Techniques and Applications
Kursuncu, U., Gaur, M., Lokala, U., Thirunarayan, K., Sheth, A., & Arpinar, I. B. (2018) · Lecture notes in social networks · doi:10.1007/978-3-319-94105-9_4 · Cited by 96 · see citing works
Hausmann, A., Toivonen, T., Heikinheimo, V., Tenkanen, H., Slotow, R., & Minin, E. D. (2017) · Scientific Reports · doi:10.1038/s41598-017-00858-6 · Cited by 94 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763 · see citing works
Ideological and Temporal Components of Network Polarization in Online Political Participatory Media
García, D., Abisheva, A., Schweighofer, S., Serdült, U., & Schweitzer, F. (2015) · Policy & Internet · doi:10.1002/poi3.82 · Cited by 91 · see citing works
The Twitter parliamentarian database: Analyzing Twitter politics across 26 countries
van Vliet, L., Törnberg, P., & Uitermark, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237073 · Cited by 93 · see citing works
A Total Error Framework for Digital Traces of Human Behavior on Online Platforms
Sen, I., Flöck, F., Weller, K., Weiß, B., & Wagner, C. (2021) · Public Opinion Quarterly · doi:10.1093/poq/nfab018 · Cited by 113 · see citing works
Broad Twitter Corpus: A Diverse Named Entity Recognition Resource
Derczynski, L., Bontcheva, K., & Roberts, I. G. (2016) · International Conference on Computational Linguistics · Cited by 86 · see citing works
Methods for Social Media Monitoring Related to Vaccination: Systematic Scoping Review
Karafillakis, E., Martin, S., Simas, C., Olsson, K., Takács, J., Dada, S., & Larson, H. J. (2020) · JMIR Public Health and Surveillance · doi:10.2196/17149 · Cited by 85 · see citing works
Seeing the smart city on Twitter: Colour and the affective territories of becoming smart
Rose, G., & Willis, A. (2018) · Environment and Planning D Society and Space · doi:10.1177/0263775818771080 · Cited by 83 · see citing works
Online Social Media in the Syria Conflict: Encompassing the Extremes and the In-Betweens
O’Callaghan, D., Prucha, N., Greene, D., Conway, M., Cunningham, P., & Carthy, J. (2014) · Dublin City University Open Access Institutional Repository (Dublin City University) · doi:10.1109/asonam.2014.6921619 · Cited by 22 · see citing works
Understanding high- and low-quality URL Sharing on COVID-19 Twitter streams
Singh, L., Bode, L., Budak, C., Kawintiranon, K., Padden, C., & Vraga, E. K. (2020) · Journal of Computational Social Science · doi:10.1007/s42001-020-00093-6 · Cited by 81 · see citing works
Understanding Popularity, Reputation, and Social Influence in the Twitter Society
García, D., Mavrodiev, P., Casati, D., & Schweitzer, F. (2017) · Policy & Internet · doi:10.1002/poi3.151 · Cited by 80 · see citing works
Chan, C., & Fu, K. (2017) · Journal of Computer-Mediated Communication · doi:10.1111/jcc4.12192 · Cited by 81 · see citing works
Li, D., Zhou, X., & Wang, M. (2018) · Cities · doi:10.1016/j.cities.2017.12.012 · Cited by 80 · see citing works
Wan, C., Shen, Q., & Choi, S. (2021) · Urban forestry & urban greening · doi:10.1016/j.ufug.2021.127172 · Cited by 86 · see citing works
Meng, J., Peng, W., Tan, P., Liu, W., Cheng, Y., & Bae, A. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.039 · Cited by 79 · see citing works
Social media analytics in hospitality and tourism
Mirzaalian, F., & Halpenny, E. A. (2019) · Journal of Hospitality and Tourism Technology · doi:10.1108/jhtt-08-2018-0078 · Cited by 79 · see citing works
Min, H., Peng, Y., Shoss, M. K., & Yang, B. (2021) · Journal of Applied Psychology · doi:10.1037/apl0000886 · Cited by 79 · see citing works
A Parsimonious Language Model of Social Media Credibility Across Disparate Events
Mitra, T., Wright, G. P., & Gilbert, É. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998351 · Cited by 80 · see citing works
Gender bias in sentiment analysis
Thelwall, M. (2018) · Online Information Review · doi:10.1108/oir-05-2017-0139 · Cited by 78 · see citing works
“Although the potential for bias in big data analyses has previously been noted (Bollier, 2010; Boyd & Crawford, 2012; Hofacker, Malthouse, & Sultan, 2016; Tufekci, 2014), the source found here is more subtle than demographic differences in the people that express opinions online and is not due to analyst bias but is a…”
Background‘I’d Double Tap That!!’: street art, graffiti, and Instagram research
MacDowall, L., & de Souza, P. (2017) · Media Culture & Society · doi:10.1177/0163443717703793 · Cited by 78 · see citing works
Non-Discriminatory Machine Learning through Convex Fairness Criteria
Goel, N., Yaghini, M., & Faltings, B. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278722 · Cited by 79 · see citing works
Navigating the technology-media-movements complex
Fominaya, C. F., & Gillan, K. (2017) · Technology, Media and Social Movements · doi:10.1080/14742837.2017.1338943 · Cited by 39 · see citing works
Guerrero-Solé, F. (2018) · Social Media + Society · doi:10.1177/2056305118808776 · Cited by 75 · see citing works
A longitudinal assessment of the persistence of twitter datasets
Zubiaga, A. (2018) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24026 · Cited by 74 · see citing works
Online sentiment towards iconic species
Fink, C., Hausmann, A., & Minin, E. D. (2019) · Biological Conservation · doi:10.1016/j.biocon.2019.108289 · Cited by 72 · see citing works
Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance
Wei, W., & Liu, L. (2024) · ACM Computing Surveys · doi:10.1145/3645102 · Cited by 88 · see citing works
Assessing reliability of social media data: lessons from mining TripAdvisor hotel reviews
Xiang, Z., Du, Q., Ma, Y., & Fan, W. (2017) · Information Technology & Tourism · doi:10.1007/s40558-017-0098-z · Cited by 73 · see citing works
Fiesler, C. L., Dye, M., Feuston, J. L., Hiruncharoenvate, C., Hutto, C. J., Morrison, S., Roshan, P. K., Pavalanathan, U., Bruckman, A. S., Choudhury, M. D., et al. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998223 · Cited by 71 · see citing works
Social scholarship revisited: Changing scholarly practices in the age of social media
Greenhow, C. M., Gleason, B., & Willet, K. B. S. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12772 · Cited by 71 · see citing works
The ecology of trust among hackers
Dupont, B., Côté, A.-M., Savine, C., & Décary-Hêtu, D. (2016) · Global Crime · doi:10.1080/17440572.2016.1157480 · Cited by 69 · see citing works
Community crosstalk: an exploratory analysis of destination and festival eWOM on Twitter
Williams, N. L., Inversini, A., Buhalis, D., & Ferdinand, N. (2015) · Journal of Marketing Management · doi:10.1080/0267257x.2015.1035308 · Cited by 68 · see citing works
Quantifying and alleviating political bias in language models
Liu, R., Jia, C., Wei, J., Xu, G., & Vosoughi, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103654 · Cited by 68 · see citing works
- Authors
- Tufekci, Z.
- Venue
- Proceedings of the International AAAI Conference on Web and Social Media
- Published
- 2014-05-16
- Volume
- 8
- Issue
- 1
- Pages
- 505-514
- ISSN
- 2162-3449 (ISSN-L)
- 2334-0770
- OpenAlex
- W2102861780
- Cited by
- 780
- References
- 27
- Citations by year
- 2026: 52 · 2025: 49 · 2024: 54 · 2023: 60 · 2022: 58 · 2021: 98 · 2020: 73 · 2019: 71 · 2018: 91 · 2017: 72 · 2016: 59 · 2015: 34 · 2014: 9
- FWCI
- 117.22
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the International AAAI Conference on Web and Social Media (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- social media big data, representativeness, validity, sampling bias, Twitter, hashtags, sampling frame, field effect, network analysis, methodological pitfalls
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
