Twitter Discussions and Emotions About the COVID-19 Pandemic: Machine Learning Approach
Open accessPublished version of a preprint
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
BACKGROUND: It is important to measure the public response to the COVID-19 pandemic. Twitter is an important data source for infodemiology studies involving public response monitoring. OBJECTIVE: The objective of this study is to examine COVID-19-related discussions, concerns, and sentiments using tweets posted by Twitter users. METHODS: We analyzed 4 million Twitter messages related to the COVID-19 pandemic using a list of 20 hashtags (eg, "coronavirus," "COVID-19," "quarantine") from March 7 to April 21, 2020. We used a machine learning approach, Latent Dirichlet Allocation (LDA), to identify popular unigrams and bigrams, salient topics and themes, and sentiments in the collected tweets. RESULTS: Popular unigrams included "virus," "lockdown," and "quarantine." Popular bigrams included "COVID-19," "stay home," "corona virus," "social distancing," and "new cases." We identified 13 discussion topics and categorized them into 5 different themes: (1) public health measures to slow the spread of COVID-19, (2) social stigma associated with COVID-19, (3) COVID-19 news, cases, and deaths, (4) COVID-19 in the United States, and (5) COVID-19 in the rest of the world. Across all identified topics, the dominant sentiments for the spread of COVID-19 were anticipation that measures can be taken, followed by mixed feelings of trust, anger, and fear related to different topics. The public tweets revealed a significant feeling of fear when people discussed new COVID-19 cases and deaths compared to other topics. CONCLUSIONS: This study showed that Twitter data and machine learning approaches can be leveraged for an infodemiology study, enabling research into evolving public discussions and sentiments during the COVID-19 pandemic. As the situation rapidly evolves, several topics are consistently dominant on Twitter, such as confirmed cases and death rates, preventive measures, health authorities and government policies, COVID-19 stigma, and negative psychological reactions (eg, fear). Real-time monitoring and assessment of Twitter discussions and concerns could provide useful data for public health emergency responses and planning. Pandemic-related fear, stigma, and mental health concerns are already evident and may continue to influence public trust when a second wave of COVID-19 occurs or there is a new surge of the current pandemic.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, most were based on tweets collected in the early stages of the pandemic, which were analyzed from an infoveillance perspective, thereby limiting their analyses to health-related responses (e.g., Abd-Alrazaq et al., 2020; Boon-Itt et al., 2020; Xue et al., 2020).”
From Understanding user responses to the COVID-19 pandemic on Twitter from a terror management theory perspective: Cultural differences among the US, UK and India (2022) · cites this paper as Xue et al., 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
S., N. L. (2017) · doi:10.1177/1609406917733847 · Cited by 14,051
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
The COVID-19 social media infodemic
Cinelli, M., Quattrociocchi, W., Galeazzi, A., Valensise, C. M., Brugnoli, E., Schmidt, A. L., Zola, P., Zollo, F., & Scala, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-73510-5 · Cited by 2,313 · cited 1×
“Existing studies have indicated the feasibility of using LDA to identify the patterns and themes of tweets related to COVID-19 [11,22].”
BackgroundExploring the Space of Topic Coherence Measures
Röder, M., Both, A., & Hinneburg, A. (2015) · Proceedings of the Eighth ACM International Conference on Web Search and Data Mining · doi:10.1145/2684822.2685324 · Cited by 2,224
Li, S., Wang, Y., Xue, J., Zhao, N., & Zhu, T. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17062032 · Cited by 1,835 · cited 2×
“Comparison With Prior Work Our findings are consistent with previous studies using social media data to assess the public health responses and sentiments related to COVID-19, and suggest that public attention has been focusing on the following topics since January 2020: (1) the confirmed cases and death rates [11,18,3…”
BackgroundResultSelf-citationPandemics in the Age of Twitter: Content Analysis of Tweets during the 2009 H1N1 Outbreak
Chew, C., & Eysenbach, G. (2010) · PLoS ONE · doi:10.1371/journal.pone.0014118 · Cited by 1,614
Signorini, A., Segre, A. M., & Polgreen, P. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019467 · Cited by 1,235
Studies in Computational Intelligence
Ortíz-Arroyo, D. (2014) · Studies in computational intelligence · Cited by 1,193
Coronavirus Goes Viral: Quantifying the COVID-19 Misinformation Epidemic on Twitter
Kouzy, R., Abi Jaoude, J., Kraitem, A., El Alam, M. B., Karam, B., Adib, E., Zarka, J., Traboulsi, C., Akl, E., & Baddour, K. (2020) · Cureus · doi:10.7759/cureus.7255 · Cited by 899
COVID-19 and the 5G Conspiracy Theory: Social Network Analysis of Twitter Data
Ahmed, W., Vidal-Alaball, J., Downing, J., & López Seguí, F. (2020) · Journal of Medical Internet Research · doi:10.2196/19458 · Cited by 797
Top concerns of tweeters during the COVID-19 pandemic: Infoveillance study
Abd-Alrazaq, A., Alhuwail, D., Househ, M., Hamdi, M., & Shah, Z. (2020) · Journal of Medical Internet Research · doi:10.2196/19016 · Cited by 774 · cited 5×
“Compared with a study examining public discussions and concerns related to COVID-19 using Twitter data from January 20 to March 7, 2020, we found that several salient topics are no longer popular: (1) an outbreak in South Korea, (2) the Diamond Princess cruise ship, (3) the economic impact [11,32], and (4) supply chai…”
BackgroundResultInfluentialChen, E., Lerman, K., & Ferrara, E. (2020) · JMIR Public Health and Surveillance · doi:10.2196/19273 · Cited by 762
Dissemination of health information through social networks: Twitter and antibiotics
Scanfeld, D., Scanfeld, V., & Larson, E. L. (2010) · American Journal of Infection Control · doi:10.1016/j.ajic.2009.11.004 · Cited by 702
Early Assessment of Anxiety and Behavioral Response to Novel Swine-Origin Influenza A(H1N1)
Jones, J. H., & Salathé, M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0008032 · Cited by 427
Rosenberg, H., Syed, S., & Rezaie, S. (2020) · CJEM · doi:10.1017/cem.2020.361 · Cited by 342
Xue, J., Chen, J., Chen, C., Zheng, C., Li, S., & Zhu, T. (2020) · PLOS ONE · doi:10.1371/journal.pone.0239441 · Cited by 332
Self-citationWorld leaders' usage of twitter in response to the COVID-19 pandemic: A content analysis
Rufai, S. R., & Bunce, C. (2020) · Journal of Public Health · doi:10.1093/pubmed/fdaa049 · Cited by 308 · cited 2×
“Comparison With Prior Work Our findings are consistent with previous studies using social media data to assess the public health responses and sentiments related to COVID-19, and suggest that public attention has been focusing on the following topics since January 2020: (1) the confirmed cases and death rates [11,18,3…”
BackgroundResultBudhwani, H., & Sun, R. (2020) · Journal of Medical Internet Research · doi:10.2196/19301 · Cited by 280
Interpretation and trust: designing model-driven visualizations for text analysis
Chuang, J., Ramage, D., Manning, C., & Heer, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207738 · Cited by 244
Medford, R. J., Saleh, S. N., Sumarsono, A., Perl, T. M., & Lehmann, C. U. (2020) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofaa258 · Cited by 242
An Overview of Sentiment Analysis in Social Media and Its Applications in Disaster Relief
Beigi, G., Hu, X., Maciejewski, R., & Liu, H. (2016) · Studies in Computational Intelligence · doi:10.1007/978-3-319-30319-2_13 · Cited by 230
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7_1 · Cited by 213
Cheong, M., & Lee, V. C. S. (2011) · Information Systems Frontiers · doi:10.1007/s10796-010-9273-x · Cited by 209
The Peru Approach against the COVID-19 Infodemic: Insights and Strategies
Alvarez-Risco, A., Mejia, C. R., Delgado-Zegarra, J., Del-Aguila-Arcentales, S., Arce-Esquivel, A. A., Valladares-Garrido, M. J., Rosas del Portal, M., Villegas, L. F., Curioso, W. H., Sekar, M. C., et al. (2020) · The American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.20-0536 · Cited by 172
The Hidden Pandemic of Family Violence During COVID-19: Unsupervised Learning of Tweets
Xue, J., Chen, J., Chen, C., Hu, R., & Zhu, T. (2020) · Journal of Medical Internet Research · doi:10.2196/24361 · Cited by 160
Self-citationMackey, T., Purushothaman, V., Li, J., Shah, N., Nali, M., Bardier, C., Liang, B., Cai, M., & Cuomo, R. (2020) · JMIR Public Health and Surveillance · doi:10.2196/19509 · Cited by 142
Xiang, X., Lü, X., Halavanau, A., Xue, J., Sun, Y., Lai, P. H. L., & Wu, Z. (2020) · The Journals of Gerontology Series B · doi:10.1093/geronb/gbaa128 · Cited by 132
Self-citationSu, Y., Xue, J., Liu, X., Wu, P., Chen, J., Chen, C., Liu, T., Gong, W., & Zhu, T. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17124552 · Cited by 122
Self-citationStokes, D. C., Andy, A., Guntuku, S. C., Ungar, L. H., & Merchant, R. M. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05889-w · Cited by 77
Kim, Y., & Kim, J. H. (2020) · Health Informatics Journal · doi:10.1177/1460458219896673 · Cited by 48
Using Data Mining Techniques to Examine Domestic Violence Topics on Twitter
Xue, J., Chen, J., & Gelles, R. J. (2019) · Violence and Gender · doi:10.1089/vio.2017.0066 · Cited by 34
Self-citationNarain, D. S. (2020) · Journal of Medical Science And clinical Research · doi:10.18535/jmscr/v8i4.63 · Cited by 4
Research works printed but not linked: 2
- Blei, D, 2003, Journal of Machine Learning Research March, 3, 993
- Kaila, RP, 2020, International Journal of Advanced Research in Engineering and Technology, 11, 128
Cited by
A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts
Egger, R., & Yu, J. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.886498 · Cited by 970
Rustam, F., Khalid, M., Aslam, W., Rupapara, V., Mehmood, A., & Choi, G. S. (2021) · PLOS ONE · doi:10.1371/journal.pone.0245909 · Cited by 313
Comparison of text preprocessing methods
Chai, C. P. (2023) · Natural Language Engineering · doi:10.1017/s1351324922000213 · Cited by 212
Applications of artificial intelligence in battling against covid-19: A literature review
Tayarani N., M.-H. (2021) · Chaos, Solitons & Fractals · doi:10.1016/j.chaos.2020.110338 · Cited by 210
Machine Learning in Healthcare Communication
Siddique, S., & Chow, J. C. L. (2021) · Encyclopedia · doi:10.3390/encyclopedia1010021 · Cited by 190
Metadata
- Authors
- Xue, J., Chen, J., Hu, R., Chen, C., Zheng, C., Su, Y., & Zhu, T.
- Affiliations
- University of Toronto (CA)
- University of Pittsburgh (US)
- Chinese Academy of Sciences (CN)
- Corresponding author
- Zhu, Tingshao
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-11-25
- Volume
- 22
- Issue
- 11
- Pages
- e20550
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7690968
- OpenAlex
- W3096451393
- Cited by
- 369
- References
- 34
- Citations by year
- 2026: 22 · 2025: 26 · 2024: 41 · 2023: 83 · 2022: 99 · 2021: 92 · 2020: 6
- FWCI
- 27.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Institutional Repository of Institute of Psychology, Chinese Academy of Sciences (Institute of Psychology, Chinese Academy of Sciences) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- TSpace (University of Toronto) (submittedVersion, cc-by)
- Greater South Information System
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Communication › Communication and COVID-19 Impact
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- COVID-19 pandemic, Twitter, public sentiment, infodemiology, social media discourse, fear, COVID-19 stigma, public health measures, social distancing, quarantine, hashtags, topic modeling, latent Dirichlet allocation, public trust
- MeSH terms
- Machine Learning, COVID-19, SARS-CoV-2, Data Collection, Emotions, Humans, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.