The DOI matches its Crossref record; 78 of the 82 references it cites were found in a registry or an index.
Tampering with Twitter’s Sample API
Open accessAbstract
Social media data is widely analyzed in computational social science. Twitter, one of the largest social media platforms, is used for research, journalism, business, and government to analyze human behavior at scale. Twitter offers data via three different Application Programming Interfaces (APIs). One of which, Twitter’s Sample API, provides a freely available 1% and a costly 10% sample of all Tweets. These data are supposedly random samples of all platform activity. However, we demonstrate that, due to the nature of Twitter’s sampling mechanism, it is possible to deliberately influence these samples, the extent and content of any topic, and consequently to manipulate the analyses of researchers, journalists, as well as market and political analysts trusting these data sources. Our analysis also reveals that technical artifacts can accidentally skew Twitter’s samples. Samples should therefore not be regarded as random. Our findings illustrate the critical limitations and general issues of big data sampling, especially in the context of proprietary data and undisclosed details about data handling.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Such accounts use tricks such as hijacking “trending” hashtags of keywords to gain visibility (Thomas et al., 2011), and can (purposefully or accidentaly) distort the statistics of datasets collected from social platforms”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Pfeffer et al., 2018
In the citing paper
“However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-related studies [35,44,45].”
From Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media (2020) · cites this paper as [41]
In the citing paper
“Recent research has also questioned the credibility of the “1% Twitter stream” aspect, noting that actual sampling data was smaller than what it ideally should have been [41].”
From Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media (2020) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,957
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
Big data: a revolution that will transform how we live, work, and think
(2013) · Choice Reviews Online · doi:10.5860/choice.50-6804 · Cited by 3,657
Earthquake shakes Twitter users
Sakaki, T., Okazaki, M., & Matsuo, Y. (2010) · International World Wide Web Conference (WWW) · doi:10.1145/1772690.1772777 · Cited by 3,631
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Twitter as a Corpus for Sentiment Analysis and Opinion Mining
Pak, A., & Paroubek, P. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/47akcj8foo4d · Cited by 2,784
Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T. O., Sandner, P., & Welpe, I. M. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,647
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523
Research works printed but not linked: 6
- Elmer G, Langlois G, Redden J (2015) Compromised data: from social media to big data. Bloomsbury Publishing, New York
- Gaffney D, Puschmann C (2013) Data collection on Twitter, pp. 55–67. Peter Lang, New York
- Neuberger C, vom Hofe J, Nuernbergk C (2014) The use of Twitter by professional journalists. results of a newsroom survey in Germany. In: Weller K, Bruns A, Burgess J, Mahrt M, Puschmann C (eds) Twitter and society. Peter Lang, New York, pp 345–357
- Elmer G, Langlois G, Redden J (2015) Compromised data: from social media to big data. Bloomsbury Publishing, USA
- Weller K, Bruns A, Burgess J, Mahrt M, Puschmann C Twitter and Society, vol. 89. P. Lang
- de Saint-Exupéry A (1943) The Little Prince. Reynal & Hitchcock, New York
Cited by
Sorted by citations
Items 101–136 of 136
Theodorakopoulos, L., & Theodoropoulou, A. (2026) · Electronics · doi:10.3390/electronics15071405 · Cited by 1
Online identity as a collective labeling process
Barron, A. T. J., ten Thij, M., & Bollen, J. (2023) · Journal of Physics Complexity · doi:10.1088/2632-072x/acc62c · Cited by 1
Federico, L., Mounim, A., Caldarelli, G., & Riotta, G. (2024) · Scientific Reports · doi:10.1038/s41598-024-65564-6 · Cited by 1
Chasing the Wrong Cloud: Mapping the 2019 Vaping Epidemic Using Data from Social Media
Gera, P., & Ciampaglia, G. L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17114-7_1 · Cited by 1
Does geographical location have an impact on data samples extracted from Twitter?
Ivanova, R. V., Sobernig, S., & Strembeck, M. (2022) · International Conference on Social Networks Analysis, Management and Security (SNAMS) · doi:10.1109/snams58071.2022.10062544 · Cited by 1
How Violence Affects ProtestsPreprint
STEINERT-THRELKELD, Z., Joo, J., & Chan, A. (2019) · doi:10.33774/apsa-2019-bv6zd · Cited by 1
Ródenas, M. J. D. L. (2019) · Hispania Sacra · doi:10.3989/hs.2019.037
Burns, B. (2020) · Digital Commons - Michigan Tech (Michigan Technological University) · doi:10.37099/mtu.dc.etdr/1078
Replicable Privacy: Enabling Replication on Sensitive Internet DataPreprint
Baleato, S., Honaker, J., & Crosas, M. (2020) · doi:10.33774/apsa-2020-mpq30
A framework for digital trace data collection through data donation.Preprint
Boeschoten, L., Ausloos, J., Møller, J., Araujo, T., & Oberski, D. L. (2020) · arXiv (Cornell University)
Analysis of complex socio-economical systems
Bokányi, E. (2019) · Eötvös Loránd Tudományegyetem · doi:10.15476/elte.2019.054
Spatially concentrated social capital of urban residentsPreprint
Kovács, Á. J., Juhász, S., Bokányi, E., & Lengyel, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13474
Developing natural language processing instruments to study sociotechnical systemsPreprint
Lab, S. (2021) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/zrbd4
Conservative Social Media Worsened COVID-19 Mortality in the US: Evidence from Swing StatesPreprint
Jensen, J. L., Karell, D., & Yntiso, S. (2022) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/edbux
Rojas, R., Álvarez-Peralta, M., & Diefenbacher, S. (2022) · Revista de la Asociación Española de Investigación de la Comunicación · doi:10.24137/raeic.9.18.5
Prediction of Real Popularity based on Sample Debias
Li, L., Gao, L., Xie, F., & Zhou, B. (2022) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc55868.2022.00072
DisKeyword: Tweet Corpora Exploration for Keyword Selection
Lévy, S., & Reihaneh Rabbany, K. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3573033
The More The Merrier: Reconstruction of Twitter Firehose
Alhosseini, S. A., & Meinel, C. (2023) · International Conference on Information Networking · doi:10.1109/icoin56518.2023.10048898
Studying Social Interactions Using Digital Trace Data
Gil‐Clavel, S. (2023) · doi:10.33612/diss.688253059
Li, L., & Liu, G. K. (2023) · International Conference on Behavioral, Economic, and Socio-Cultural Computing · doi:10.1109/besc59560.2023.10386565
Wieczorek, O., & Heiberger, R. H. (2024) · Netzwerkforschung · doi:10.1007/978-3-658-37507-2_38-1
Ismail, H. (2025) · Journalism and Media · doi:10.3390/journalmedia6010028
Sentiment, Volume, and Topics in University Tweets: Methodology, Insights, and Challenges
Câmpan, A., Truţă, T. M., & Karki, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-92178-0_6
Wieczorek, O., & Heiberger, R. H. (2025) · Netzwerkforschung · doi:10.1007/978-3-658-37503-4_38
Social Network Analysis in Human Geography
Gong, X., Lu, Y., & Lin, Y. (2025) · Springer geography · doi:10.1007/978-3-031-87421-5_7
Climate Disasters and Risks in Online Expressions in South Africa
Ganduri, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11108-1_19
Bai, D., & Gu, Y. (2026) · Social Science Computer Review · doi:10.1177/08944393251413277
Masis, T., Eggleston, C., Green, L., Jones, T., Armstrong, M. E., & O'Connor, B. (2026) · American Speech · doi:10.1215/00031283-12519827
The Economics of Rural Development in the United States
Hill, A. E., Low, S. A., & Taylor, K. (2026) · Annual Review of Resource Economics · doi:10.1146/annurev-resource-120324-050230
Where You Are Is What You Get! Inconsistencies of Digital Trace Data across Download Locations
Hölzl, J., Keusch, F., & Collins, J. T. (2026) · Social Science Computer Review · doi:10.1177/08944393261469494
Gong, X., Liu, L., Lu, Y., Zhang, G., Huang, X., & Lin, Y. (2026) · PLoS ONE · doi:10.1371/journal.pone.0356547
Ghoroghi, A., Hodorog, A., Rezgui, Y., Ghaemi, A., & Nardi, C. D. (2026) · Progress in Disaster Science · doi:10.1016/j.pdisas.2026.100715
Metadata
- Authors
- Pfeffer, J., Mayer, K., & Morstatter, F.
- Affiliations
- Technical University of Munich (DE)
- University of Southern California (US)
- Corresponding author
- Pfeffer, Jürgen
- Venue
- EPJ Data Science
- Published
- 2018-12
- Volume
- 7
- Issue
- 1
- ISSN
- 2193-1127 (ISSN-L)
- OpenAlex
- W2904605268
- Cited by
- 135
- References
- 82
- Citations by year
- 2026: 8 · 2025: 10 · 2024: 7 · 2023: 22 · 2022: 22 · 2021: 25 · 2020: 31 · 2019: 10
- FWCI
- 23.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- EPJ Data Science (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- Funders
- Deutsche Forschungsgemeinschaft
- Technische Universität München
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- sampling bias, tweets, data manipulation, random sampling, technical artifacts, proprietary data, social media data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
