Using Twitter for Demographic and Social Science Research: Tools for Data Collection and Processing
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Despite recent and growing interest in using Twitter to examine human behavior and attitudes, there is still significant room for growth regarding the ability to leverage Twitter data for social science research. In particular, gleaning demographic information about Twitter users—a key component of much social science research—remains a challenge. This article develops an accurate and reliable data processing approach for social science researchers interested in using Twitter data to examine behaviors and attitudes, as well as the demographic characteristics of the populations expressing or engaging in them. Using information gathered from Twitter users who state an intention to not vote in the 2012 presidential election, we describe and evaluate a method for processing data to retrieve demographic information reported by users that is not encoded as text (e.g., details of images) and evaluate the reliability of these techniques. We end by assessing the challenges of this data collection strategy and discussing how large-scale social media data may benefit demographic researchers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Presentation of Self in Everyday Life.
Naegele, K. D., & Goffman, E. (1956) · American Sociological Review · doi:10.2307/2089106 · Cited by 25,991
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience
Marwick, A. E., & boyd, D. (2011) · New Media & Society · doi:10.1177/1461444810365313 · Cited by 4,612
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Sensitive questions in surveys.
Tourangeau, R., & Yan, T. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.5.859 · Cited by 3,057
Why we twitter: understanding microblogging usage and communities
Java, A., Song, X., Finin, T., & Tseng, B. (2007) · Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis · doi:10.1145/1348549.1348556 · Cited by 2,645 · cited 1×
“This concise format allows users to update their ‘‘blogs’’ multiple times per day, rather than every few days, as is the case with traditional blogging platforms (Java et al. 2007).”
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
Snow, R., O'Connor, B., Jurafsky, D., & Ng, A. Y. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613751 · Cited by 1,887 · cited 2×
“In research experimenting directly with Turkers, Snow et al. (2008) found that in regard to many language processing tasks such as affect determination, Turkers are just as effective as and less expensive than expert labelers.”
Friend networking sites and their relationship to adolescents’ well-being and social self-esteem
Valkenburg, P. M., Peter, J., & Schouten, A. P. (2006) · CyberPsychology & Behavior · doi:10.1089/cpb.2006.9.584 · Cited by 1,531
Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures
Golder, S. A., & Macy, M. W. (2011) · Science · doi:10.1126/science.1202775 · Cited by 1,155 · cited 4×
“Web-based data collection has gained prominence among social science researchers as a means of collecting large amounts of data to explore topics such as election forecasting, tracking social trends, and time usage (Golder and Macy 2011; Naaman et al. 2011; Tumasjan et al. 2010).”
InfluentialThe viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,018 · cited 2×
“Behrend et al. (2011), for example, distributed a short survey to both the Turkers and a sample of university students as a means of comparing the psychometric properties of each.”
Understanding the Demographics of Twitter Users
Mislove, A., Lehmann, S., Ahn, Y.-Y., Onnela, J.-P., & Rosenquist, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14168 · Cited by 759
Unanticipated Gains: Origins of Network Inequality in Everyday Life
Small, M. L. (2009) · doi:10.1093/acprof:oso/9780195384352.001.0001 · Cited by 732
Dynamic debates: an analysis of group polarization over time on Twitter
Yardi, S., & Boyd, D. (2010) · Bulletin of Science, Technology & Society · doi:10.1177/0270467610380011 · Cited by 625
Social desirability bias in voter turnout reports: Tests using the item count technique
Holbrook, A. L., & Krosnick, J. A. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfp065 · Cited by 539
Unanticipated gains: origins of network inequality in everyday life
(2010) · Choice Reviews Online · doi:10.5860/choice.47-2900 · Cited by 528
Characterizing debate performance via aggregated twitter sentiment
Diakopoulos, N. A., & Shamma, D. A. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753504 · Cited by 470
Election Forecasts With Twitter: How 140 Characters Reflect the Political Landscape
Tumasjan, A., Sprenger, T. O., Sandner, P. G., & Welpe, I. M. (2011) · Social Science Computer Review · doi:10.1177/0894439310386557 · Cited by 447
Predicting Flu Trends using Twitter data
Achrekar, H., Gandhe, A., Lazarus, R., Yu, S.-H., & Liu, B. (2011) · 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infcomw.2011.5928903 · Cited by 442
Hip and trendy: Characterizing emerging trends on Twitter
Naaman, M., Becker, H., & Gravano, L. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21489 · Cited by 322 · cited 1×
“Web-based data collection has gained prominence among social science researchers as a means of collecting large amounts of data to explore topics such as election forecasting, tracking social trends, and time usage (Golder and Macy 2011; Naaman et al. 2011; Tumasjan et al. 2010).”
De Longueville, B., Smith, R. S., & Luraschi, G. (2009) · Proceedings of the 2009 International Workshop on Location Based Social Networks · doi:10.1145/1629890.1629907 · Cited by 307
Not by the Book: Facebook as a Sampling Frame
Brickman Bhutta, C. (2012) · Sociological Methods & Research · doi:10.1177/0049124112440795 · Cited by 290
Reducing Vote Overreporting in Surveys: Social Desirability, Memory Failure, and Source Monitoring
Belli, R. F., Traugott, M. W., Young, M., & McGonagle, K. A. (1999) · Public Opinion Quarterly · doi:10.1086/297704 · Cited by 241 · cited 3×
“It may be utilized, for example, by researchers who seek to examine large-scale processes of contagion, track preferences, and/ or opinions among broad audiences; examine behaviors and attitudes where social desirability bias in an official survey may occur (e.g., racist attitudes, voting behavior or anti-immigrant se…”
InfluentialTwitter: The Electoral Connection?
Lassen, D. S., & Brown, A. R. (2011) · Social Science Computer Review · doi:10.1177/0894439310382749 · Cited by 236
Public Health Surveillance of Dental Pain via Twitter
Heaivilin, N., Gerbert, B., Page, J. E., & Gibbs, J. L. (2011) · Journal of Dental Research · doi:10.1177/0022034511415273 · Cited by 229 · cited 1×
“As such, they have sparked the interest of many researchers seeking to better understanding social relationships and behavior (Brickman Bhutta 2012; Golder and Macy 2011; Heaivilin et al. 2011; Lowe et al. 2012; Moreno et al. 2012; Valkenburg, Peter, and Schouten 2006).”
Using the Amazon Mechanical Turk for transcription of spoken language
Marge, M., Banerjee, S., & Rudnicky, A. I. (2010) · 2010 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2010.5494979 · Cited by 224
Janus, A. L. (2010) · Social Science Quarterly · doi:10.1111/j.1540-6237.2010.00742.x · Cited by 218 · cited 1×
“…an official survey may occur (e.g., racist attitudes, voting behavior or anti-immigrant sentiments; Belli et al. 1999; Holbrook and Krosnick 2010; Janus 2010; Tourangeau and Yan 2007); analyze collective experiences based on a timely event (e.g., terrorist attacks or natural disasters; Cassa et…”
The Impact of Physical Attractiveness on Achievement and Psychological Well-Being
Umberson, D., & Hughes, M. (1987) · Social Psychology Quarterly · doi:10.2307/2786823 · Cited by 192
Twitter Archives and the Challenges of "Big Social Data" for Media and Communication Research
Burgess, J., & Bruns, A. (2012) · M/C Journal · doi:10.5204/mcj.561 · Cited by 150
Twitter as a Sentinel in Emergency Situations: Lessons from the Boston Marathon Explosions
Cassa, C., Chunara, R., Mandl, K., & Brownstein, J. S. (2013) · PLoS Currents · doi:10.1371/currents.dis.ad70cd1c8bc585e9470046cde334ee4b · Cited by 129 · cited 1×
“…2010; Janus 2010; Tourangeau and Yan 2007); analyze collective experiences based on a timely event (e.g., terrorist attacks or natural disasters; Cassa et al. 2013; Sutton et al. 2012); gather large amounts of data on hard-toreach populations (Koepfler and Fleischmann 2012); and potentially to…”
Guest Editorial: Big Social Data Analysis
Cambria, E., Wang, H., & White, B. (2014) · Knowledge-Based Systems · doi:10.1016/j.knosys.2014.07.002 · Cited by 125 · cited 1×
“The ability to aggregate vast amounts of digital traces of human behavior through social media platforms represents a new data collection paradigm for social science research (Cambria, Wang, and White 2014; Lazer et al. 2009).”
The cultural grounding of personal relationship: The importance of attractiveness in everyday life.
Anderson, S. L., Adams, G., & Plaut, V. C. (2008) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.95.2.352 · Cited by 112
Mining twitter: A source for psychological wisdom of the crowds
Reips, U., & Garaizar, P. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0116-6 · Cited by 83
College Students’ Alcohol Displays on Facebook: Intervention Considerations
Moreno, M. A., Grant, A., Kacvinsky, L., Egan, K. G., & Fleming, M. F. (2012) · Journal of American College Health · doi:10.1080/07448481.2012.663841 · Cited by 58 · cited 1×
“As such, they have sparked the interest of many researchers seeking to better understanding social relationships and behavior (Brickman Bhutta 2012; Golder and Macy 2011; Heaivilin et al. 2011; Lowe et al. 2012; Moreno et al. 2012; Valkenburg, Peter, and Schouten 2006).”
Why watching movie tweets won't tell the whole story?
Wong, F. M. F., Sen, S., & Chiang, M. (2012) · doi:10.1145/2342549.2342564 · Cited by 53
Crowdsourcing Preference Judgments for Evaluation of Music Similarity Tasks
Urbano, J., Morato, J., Marrero, M., & Martín, D. R. (2010) · Cited by 45
Connected communications: Network structures of official communications in a technological disaster.
Sutton, J. N., Spiro, E. S., Johnson, B., Fitzhugh, S. M., Greczek, M., & Butts, C. T. (2012) · ISCRAM · Cited by 33
Self-citationStudying the values of hard-to-reach populations
Koepfler, J. A., & Fleischmann, K. R. (2012) · iConference · doi:10.1145/2132176.2132183 · Cited by 31
Investigating the use of social media to help women from going back to smoking post‐partum
Lowe, J. B., Barnes, M., Teo, C., & Sutherns, S. (2012) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1753-6405.2012.00826.x · Cited by 29 · cited 1×
“As such, they have sparked the interest of many researchers seeking to better understanding social relationships and behavior (Brickman Bhutta 2012; Golder and Macy 2011; Heaivilin et al. 2011; Lowe et al. 2012; Moreno et al. 2012; Valkenburg, Peter, and Schouten 2006).”
Cited by
Computational Social Science and Sociology
Edelmann, A., Wolff, T., Montagne, D., & Bail, C. A. (2020) · Annual Review of Sociology · doi:10.1146/annurev-soc-121919-054621 · Cited by 305
“…demographic characteristics of neighborhoods (Gebru et al. 2017), crowd-sourcing techniques to measure demographics of social media populations (McCormick et al. 2017), websites to map large-scale genealogy trees (Kaplanis et al. 2018), and online image data to predict age (Helleringer et al.…”
Open Humans: a platform for participant-centered research and personal data exploration
Greshake Tzovaras, B., Angrist, M., Arvai, K., Dulaney, M., Estrada-Galiñanes, V., Gunderson, B., Head, T., Lewis, D., Nov, O., Shaer, O., et al. (2019) · GigaScience · doi:10.1093/gigascience/giz076 · Cited by 56
Hswen, Y., Naslund, J. A., Brownstein, J. S., & Hawkins, J. B. (2018) · Psychiatric Quarterly · doi:10.1007/s11126-017-9559-y · Cited by 48
Measures of the Degree of Departure from Ignorable Sample Selection
Little, R. J. A., West, B. T., Boonstra, P. S., & Hu, J. (2020) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smz023 · Cited by 47
Hswen, Y., Naslund, J. A., Brownstein, J. S., & Hawkins, J. B. (2018) · JMIR Mental Health · doi:10.2196/11483 · Cited by 43
“This highlights an immense source of unsolicited data with exciting potential to study various aspects of human behavior and monitor health conditions including mental illness [25].”
Background
Metadata
- Authors
- McCormick, T. H., Lee, H., Cesare, N., Shojaie, A., & Spiro, E. S.
- Venue
- Sociological Methods & Research
- Publisher
- SAGE Publications
- Published
- 2015-10-09
- Volume
- 46
- Issue
- 3
- Pages
- 390-421
- ISSN
- 0049-1241 (ISSN-L)
- 1552-8294
- OpenAlex
- W2337647281
- Cited by
- 207
- References
- 40
- FWCI
- 27.14
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.