CoChat

Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media

Open access
Saha, Koustuv, Torous, John, Caine, Eric D, De Choudhury, Munmun · 2020 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index; 10 of the 71 sources its claims cite were checked for support.

Abstract

Background: The COVID-19 pandemic has caused several disruptions in personal and collective lives worldwide. The uncertainties surrounding the pandemic have also led to multifaceted mental health concerns, which can be exacerbated with precautionary measures such as social distancing and self-quarantining, as well as societal impacts such as economic downturn and job loss. Despite noting this as a “mental health tsunami”, the psychological effects of the COVID-19 crisis remain unexplored at scale. Consequently, public health stakeholders are currently limited in identifying ways to provide timely and tailored support during these circumstances. Objective: Our study aims to provide insights regarding people’s psychosocial concerns during the COVID-19 pandemic by leveraging social media data. We aim to study the temporal and linguistic changes in symptomatic mental health and support expressions in the pandemic context. Methods: We obtained about 60 million Twitter streaming posts originating from the United States from March 24 to May 24, 2020, and compared these with about 40 million posts from a comparable period in 2019 to attribute the effect of COVID-19 on people’s social media self-disclosure. Using these data sets, we studied people’s self-disclosure on social media in terms of symptomatic mental health concerns and expressions of support. We employed transfer learning classifiers that identified the social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support). We then examined the changes in psychosocial expressions over time and language, comparing the 2020 and 2019 data sets. Results: We found that all of the examined psychosocial expressions have significantly increased during the COVID-19 crisis—mental health symptomatic expressions have increased by about 14%, and support expressions have increased by about 5%, both thematically related to COVID-19. We also observed a steady decline and eventual plateauing in these expressions during the COVID-19 pandemic, which may have been due to habituation or due to supportive policy measures enacted during this period. Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis. Conclusions: We studied the psychosocial effects of the COVID-19 crisis by using social media data from 2020, finding that people’s mental health symptomatic and support expressions significantly increased during the COVID-19 period as compared to similar data from 2019. However, this effect gradually lessened over time, suggesting that people adapted to the circumstances and their “new normal.” Our linguistic analyses revealed that people expressed mental health concerns regarding personal and professional challenges, health care and precautionary measures, and pandemic-related awareness. This study shows the potential to provide insights to mental health care and stakeholders and policy makers in planning and implementing measures to mitigate mental health risks amid the health crisis.

Overview

CoChat Summary

TL;DR

This quasi-experimental Twitter study compared ~59 million US posts from March 24–May 24, 2020 with ~40.9 million posts from the same period in 2019, finding significant increases in symptomatic mental health and support expressions during COVID-19. The expressions were thematically tied to the pandemic and generally declined or plateaued over the two-month treatment period.

Key findings

  • All examined psychosocial expressions increased significantly in 2020 versus 2019: symptomatic mental health expressions rose about 14% overall and support expressions about 5% overall.
  • Specific increases were anxiety 21.32%, suicidal ideation 19.73%, depression 10.18%, stress 3.76%, emotional support 4.77%, and informational support 4.78%; all had significant t tests and Cohen d > 0.4.
  • Temporal plots showed a negative slope in the treatment year (average slope = –0.03) versus a near-zero slope in control (average slope = 3.19×10⁻⁴), suggesting expressions leveled out over time.
  • SAGE language analyses found treatment-period keywords such as covid19, coronavirus, social distancing, stayathome isolation, essential workers, dr fauci, masks, ppe, furloughed, and losing jobs, while control-period keywords included hospitality, trainee, crimes, traffic, and parttime.

Methods

  • Quasi-experimental design using Twitter Streaming API data: treatment = 59,096,694 US English posts from March 24–May 24, 2020; control = 40,875,185 US English posts from March 24–May 24, 2019.
  • Transfer-learning SVM classifiers (n-grams n=1,2,3) identified anxiety, depression, stress, suicidal ideation, emotional support, and informational support; classifiers were trained on Reddit data and reported ~0.90 average accuracy, 87% Twitter agreement with expert appraisal, and support classifier k-fold accuracies of 0.71 and 0.77.
  • Analyses included two-sample t tests and Cohen d, z-score standardization using an additional year-long Twitter data set of over 240 million posts (September 2018–August 2019), SAGE for salient n-grams, and log-likelihood ratios for keyword overlap.

Limitations

  • Stated: the Twitter Streaming API lacks transparency, the “1% Twitter stream” credibility has been questioned, and actual sampling may be smaller; the authors avoided regional comparisons and absolute-number claims, filtered English-only posts, and excluded data after May 24, 2020 to avoid confounding with George Floyd/Black Lives Matter events.
  • Stated: Twitter data have self-selection and representation biases, the study is observational and does not measure true causality, and support expressions may include both support-seeking and support-providing.
  • Evident from the text: mental health and support labels are machine-predicted proxies transferred from Reddit rather than clinical diagnoses, and findings are limited to one platform and a finite two-month period.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/22600

Cites

100 of 100 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Figures

Figure 1 Comparison of symptomatic mental health expressions on social media posts in the same period (March 24 to May 24) in 2019 and 2020 (COVID-19 outbreak year). Idn.: ideation.
Figure 1 Comparison of symptomatic mental health expressions on social media posts in the same period (March 24 to May 24) in 2019 and 2020 (COVID-19 outbreak year). Idn.: ideation.CC BY · PubMed Central
Figure 2 Comparison of support expressions on social media posts in the same period (March 24 to May 24) in 2019 and 2020 (COVID-19 outbreak year).
Figure 2 Comparison of support expressions on social media posts in the same period (March 24 to May 24) in 2019 and 2020 (COVID-19 outbreak year).CC BY · PubMed Central

Verification

Claims the paper makes

8 Valid · 0 Not valid · 46 Not checked · 53 Own results and interpretations

45 more claims

Create a free account to see all 107 claims and to verify the 61 references still open.

See all claims — free account
  • It may state two separate claims as one. 37
  • Probably not a substantial claim. 34

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 100 references

Research works printed but not linked: 5

  1. SadilekAKautzHSilenzioVPredicting disease transmission from geo-tagged micro-blog data2012Twenty-Sixth AAAI Conference on Artificial IntelligenceJuly 2012Toronto, ON
  2. CoppersmithGAHarmanCTDredzeMHMeasuring post traumatic stress disorder in Twitter2014Eighth International AAAI Conference on Weblogs and Social Media2014Ann Arbor, MI
  3. Saha, K, 2018, Proc Int AAAI Conf Weblogs Soc Media, 2018, 320
  4. EisensteinJAhmedAXingEPSparse additive generative models of text201128th International Conference on International Conference on Machine LearningJune 2011Bellevue, WA
  5. Aassve, A, 2020, Working Papers 661, Innocenzo Gasparini Institute for Economic Research, 1

Cited by

See all 182

Metadata

Authors
Saha, K., Torous, J., Caine, E. D., & De Choudhury, M.
Affiliations
  • Georgia Institute of Technology (US)
  • Beth Israel Deaconess Medical Center (US)
  • University of Rochester Medicine (US)
Corresponding author
Saha, Koustuv
Venue
Journal of Medical Internet Research
Published
2020-11-24
Volume
22
Issue
11
Pages
e22600
ISSN
1438-8871 (ISSN-L)
OpenAlex
W3097203256
Cited by
166
References
95
Citations by year
2026: 11 · 2025: 20 · 2024: 20 · 2023: 30 · 2022: 57 · 2021: 28
FWCI
31.43
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Science Foundation (2027689)
  • National Institute of Mental Health (R01MH117172)
  • National Institute of General Medical Sciences (R01GM112697)
Topics
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Social Sciences › Psychology › Clinical Psychology › COVID-19 and Mental Health
  • Social Sciences › Communication › Communication and COVID-19 Impact
Keywords
COVID-19 pandemic, mental health, social media self-disclosure, Twitter, anxiety, depression, stress, suicidal ideation, emotional support, information support, temporal trends, linguistic analysis, habituation, social distancing, public health policy, United States
MeSH terms
COVID-19, SARS-CoV-2, Humans, Psychology, Social Media, Non-Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media. https://app.cochat.ai/papers/10.2196/22600
The paper
Saha, K., Torous, J., Caine, E. D., & De Choudhury, M. (2020). Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media. Journal of Medical Internet Research. https://doi.org/10.2196/22600

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.