Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media
Open accessPublished version of a preprint
The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index; 10 of the 71 sources its claims cite were checked for support.
Abstract
Background: The COVID-19 pandemic has caused several disruptions in personal and collective lives worldwide. The uncertainties surrounding the pandemic have also led to multifaceted mental health concerns, which can be exacerbated with precautionary measures such as social distancing and self-quarantining, as well as societal impacts such as economic downturn and job loss. Despite noting this as a “mental health tsunami”, the psychological effects of the COVID-19 crisis remain unexplored at scale. Consequently, public health stakeholders are currently limited in identifying ways to provide timely and tailored support during these circumstances. Objective: Our study aims to provide insights regarding people’s psychosocial concerns during the COVID-19 pandemic by leveraging social media data. We aim to study the temporal and linguistic changes in symptomatic mental health and support expressions in the pandemic context. Methods: We obtained about 60 million Twitter streaming posts originating from the United States from March 24 to May 24, 2020, and compared these with about 40 million posts from a comparable period in 2019 to attribute the effect of COVID-19 on people’s social media self-disclosure. Using these data sets, we studied people’s self-disclosure on social media in terms of symptomatic mental health concerns and expressions of support. We employed transfer learning classifiers that identified the social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support). We then examined the changes in psychosocial expressions over time and language, comparing the 2020 and 2019 data sets. Results: We found that all of the examined psychosocial expressions have significantly increased during the COVID-19 crisis—mental health symptomatic expressions have increased by about 14%, and support expressions have increased by about 5%, both thematically related to COVID-19. We also observed a steady decline and eventual plateauing in these expressions during the COVID-19 pandemic, which may have been due to habituation or due to supportive policy measures enacted during this period. Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis. Conclusions: We studied the psychosocial effects of the COVID-19 crisis by using social media data from 2020, finding that people’s mental health symptomatic and support expressions significantly increased during the COVID-19 period as compared to similar data from 2019. However, this effect gradually lessened over time, suggesting that people adapted to the circumstances and their “new normal.” Our linguistic analyses revealed that people expressed mental health concerns regarding personal and professional challenges, health care and precautionary measures, and pandemic-related awareness. This study shows the potential to provide insights to mental health care and stakeholders and policy makers in planning and implementing measures to mitigate mental health risks amid the health crisis.
Overview
CoChat Summary
TL;DR
This quasi-experimental Twitter study compared ~59 million US posts from March 24–May 24, 2020 with ~40.9 million posts from the same period in 2019, finding significant increases in symptomatic mental health and support expressions during COVID-19. The expressions were thematically tied to the pandemic and generally declined or plateaued over the two-month treatment period.
Key findings
- All examined psychosocial expressions increased significantly in 2020 versus 2019: symptomatic mental health expressions rose about 14% overall and support expressions about 5% overall.
- Specific increases were anxiety 21.32%, suicidal ideation 19.73%, depression 10.18%, stress 3.76%, emotional support 4.77%, and informational support 4.78%; all had significant t tests and Cohen d > 0.4.
- Temporal plots showed a negative slope in the treatment year (average slope = –0.03) versus a near-zero slope in control (average slope = 3.19×10⁻⁴), suggesting expressions leveled out over time.
- SAGE language analyses found treatment-period keywords such as covid19, coronavirus, social distancing, stayathome isolation, essential workers, dr fauci, masks, ppe, furloughed, and losing jobs, while control-period keywords included hospitality, trainee, crimes, traffic, and parttime.
Methods
- Quasi-experimental design using Twitter Streaming API data: treatment = 59,096,694 US English posts from March 24–May 24, 2020; control = 40,875,185 US English posts from March 24–May 24, 2019.
- Transfer-learning SVM classifiers (n-grams n=1,2,3) identified anxiety, depression, stress, suicidal ideation, emotional support, and informational support; classifiers were trained on Reddit data and reported ~0.90 average accuracy, 87% Twitter agreement with expert appraisal, and support classifier k-fold accuracies of 0.71 and 0.77.
- Analyses included two-sample t tests and Cohen d, z-score standardization using an additional year-long Twitter data set of over 240 million posts (September 2018–August 2019), SAGE for salient n-grams, and log-likelihood ratios for keyword overlap.
Limitations
- Stated: the Twitter Streaming API lacks transparency, the “1% Twitter stream” credibility has been questioned, and actual sampling may be smaller; the authors avoided regional comparisons and absolute-number claims, filtered English-only posts, and excluded data after May 24, 2020 to avoid confounding with George Floyd/Black Lives Matter events.
- Stated: Twitter data have self-selection and representation biases, the study is observational and does not measure true causality, and support expressions may include both support-seeking and support-providing.
- Evident from the text: mental health and support labels are machine-predicted proxies transferred from Reddit rather than clinical diagnoses, and findings are limited to one platform and a finite two-month period.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
8 Valid · 0 Not valid · 46 Not checked · 53 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Fear and anxiety about a disease as infectious as COVID-19 can trigger new-onset or exacerbate existing mental illness [8].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Our copy supports only the 'new-onset' half, and at a weaker, hedged level: the Abstract lists 'fear of contracting the disease' among the stressors COVID-19 introduced to mental health, and the Discussion (p. 1) says the Kessler 6 measure 'has been shown to accurately predict serious mental illness, suggesting acute distress during COVID-19 may transfer to longer-term psychiatric disorders.' Nothing in our copy addresses exacerbation of pre-existing mental illness, and the paper itself reports cross-sectional prevalence estimates (13.6% serious psychological distress in April 2020 vs 3.9% in 2018) rather than demonstrating that fear or anxiety triggers psychiatric disorders. The citing sentence also states as a general capability what the source frames as a suggestion ('may'). (LLM, conf=0.72)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Social media provides a real-time platform where people often candidly self-disclose their concerns, opinions, and struggles during this pandemic [23].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [23] sourceExperimental — accuracy not yet measured
The section “Utilization of Data to Understand Disparities” says people may voice opinions on social media that surveys do not capture, and that publicly shared posts may reveal mistrust or preferences. It does not establish that people often candidly disclose concerns and struggles during COVID-19, or describe social media specifically as a real-time platform. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
The Abstract identifies mental-health stressors beyond infection, and the Results report substantially more serious psychological distress among US adults in April 2020 than in 2018. The paper does not compare the crisis’s overall practical impact with the number of infections or deaths, so it does not establish the claimed ‘far greater’ magnitude. (LLM, conf=0.82)
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
The Introduction says the pandemic’s direct and indirect psychological and social effects are pervasive, and the section on lockdown and social isolation says the whole population is affected in different ways. This supports an impact beyond infections and deaths, but the paper does not establish the claim’s comparison that the practical impact is “far greater” than the number of cases or fatalities. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The observations reflected that the COVID-19 pandemic has increased people’s mental health expressions on social media, aligning with other contemporary literature and media reports [8,53].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Results > Temporal Variation
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Our copy of McGinty et al. (JAMA 2020) supports the general substance — the Discussion (Methods/Discussion section) states 'The prevalence of reported symptoms of psychological distress among US adults was higher in 2020 during the COVID-19 pandemic than in 2018' (13.6% vs 3.9% serious psychological distress, Results/Figure). However, the source measures distress and loneliness via the Johns Hopkins COVID-19 Civic Life and Public Health Survey on NORC's AmeriSpeak panel and the 2018 NHIS — administered online/in household interviews as structured survey instruments (Methods). No passage anywhere in our copy addresses social media, online posts, or 'mental health expressions' on platforms, so the specifically social-media component of the citing sentence is not attributable to this paper. (LLM, conf=0.70)
- Could not verifycites [53] sourceExperimental — accuracy not yet measured
Our copy of the Bedford et al. Lancet Comment is a STAG-IH policy piece on COVID-19 risk assessment and control measures (opening section, p.1; response-measure paragraphs and recommendations; research-gaps panel); none of these passages mention mental health, psychological distress, or social-media postings. Targeted searches ('mental health social media expressions' and 'psychological distress stigma online communication public awareness') returned only the same control/epidemiology passages plus the reference list, which cites WHO situation reports, the China CDC 72 314-case series, viral-load studies and social-distancing notices — no mental-health or social-media source. The paper therefore neither supports nor contradicts the attributed point; it simply does not address it. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis.”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 107 claims and to verify the 61 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 34
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Sample sizes ranged from two individuals in a number of case series ( Kapilan, 2020; Mamun et al., 2020b; Pirnia et al., 2020; Sahoo et al., 2020) to 60 million Twitter posts ( Saha et al., 2020).”
From The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review (2021) · cites this paper as Saha et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“The two studies we identified ( Low et al., 2020; Saha et al., 2020) reported heightened levels of suicide-related posting/suicidality.”
From The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review (2021) · cites this paper as Saha et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Methods > Examining Psychosocial Expressions Over Time and Language
“LLRs<1 indicated the greater frequency in symptomatic expressions, and LLRs>1 indicated the greater frequency in support expressions.”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Results > Linguistic Expressions
“dKeywords with LLRs<0 distinctly occurred in support expressions.”
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
The need to belong: Desire for interpersonal attachments as a fundamental human motivation.
Baumeister, R. F., & Leary, M. R. (1995) · Psychological Bulletin · doi:10.1037//0033-2909.117.3.497 · Cited by 23,245
Stress, social support, and the buffering hypothesis
Cohen, S., & Wills, T. A. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.98.2.310 · Cited by 18,093
Holmes, E. A., O'Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., Ballard, C., Christensen, H., Cohen Silver, R., Everall, I., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30168-1 · Cited by 5,715 · cited 1×
“Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9].”
Backgroundboyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479 · cited 1×
“However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-rel…”
MethodMental health and the COVID-19 pandemic
Pfefferbaum, B., & North, C. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2008017 · Cited by 4,378 · cited 2×
“Despite concerns about the myriad of social and behavioral issues associated with the COVID-19 pandemic [15,16], there has been scant research to examine its psychosocial impacts or how to predict and mitigate them.”
BackgroundPappa, S., Ntella, V., Giannakas, T., Giannakoulis, V. G., Papoutsi, E., & Katsaounou, P. (2020) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2020.05.026 · Cited by 3,617 · cited 1×
“The published literature posits that the distress and anxiety among individuals in this COVID-19 pandemic may increase the incidence of mental disorders [53,54,99]; data thus far from the United States point to a population increase in psychological distress of 10% compared to 2018 data [8], a trend that is in line wi…”
ResultMental health problems and social media exposure during COVID-19 outbreak
Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231924 · Cited by 2,830
Mental Health Problems and Social Media Exposure During COVID-19 Outbreak
Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3541120 · Cited by 2,830
Research works printed but not linked: 5
- SadilekAKautzHSilenzioVPredicting disease transmission from geo-tagged micro-blog data2012Twenty-Sixth AAAI Conference on Artificial IntelligenceJuly 2012Toronto, ON
- CoppersmithGAHarmanCTDredzeMHMeasuring post traumatic stress disorder in Twitter2014Eighth International AAAI Conference on Weblogs and Social Media2014Ann Arbor, MI
- Saha, K, 2018, Proc Int AAAI Conf Weblogs Soc Media, 2018, 320
- EisensteinJAhmedAXingEPSparse additive generative models of text201128th International Conference on International Conference on Machine LearningJune 2011Bellevue, WA
- Aassve, A, 2020, Working Papers 661, Innocenzo Gasparini Institute for Economic Research, 1
Cited by
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Self-citationZhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581318 · Cited by 288
Social media use in disaster recovery: A systematic literature review
Ogie, R. I., James, S., Moore, A., Dilworth, T., Amirghasemi, M., & Whittaker, J. (2022) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2022.102783 · Cited by 169
John, A., Eyles, E., Webb, R. T., Okolie, C., Schmidt, L., Arensman, E., Hawton, K., O'Connor, R. C., Kapur, N., Moran, P., et al. (2021) · F1000Research · doi:10.12688/f1000research.25522.2 · Cited by 189
The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: a living systematic review
John, A., Okolie, C., Eyles, E., Webb, R. T., Schmidt, L., McGuinness, L. A., Olorisade, B. K., Arensman, E., Hawton, K., Kapur, N., et al. (2020) · F1000Research · doi:10.12688/f1000research.25522.1 · Cited by 178
Metadata
- Authors
- Saha, K., Torous, J., Caine, E. D., & De Choudhury, M.
- Affiliations
- Georgia Institute of Technology (US)
- Beth Israel Deaconess Medical Center (US)
- University of Rochester Medicine (US)
- Corresponding author
- Saha, Koustuv
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-11-24
- Volume
- 22
- Issue
- 11
- Pages
- e22600
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7690250
- OpenAlex
- W3097203256
- Semantic Scholar
- b314968db7715d78c9bc4c73972f8311d477728a
- Cited by
- 166
- References
- 95
- Citations by year
- 2026: 11 · 2025: 20 · 2024: 20 · 2023: 30 · 2022: 57 · 2021: 28
- FWCI
- 31.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (2027689)
- National Institute of Mental Health (R01MH117172)
- National Institute of General Medical Sciences (R01GM112697)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › COVID-19 and Mental Health
- Social Sciences › Communication › Communication and COVID-19 Impact
- Keywords
- COVID-19 pandemic, mental health, social media self-disclosure, Twitter, anxiety, depression, stress, suicidal ideation, emotional support, information support, temporal trends, linguistic analysis, habituation, social distancing, public health policy, United States
- MeSH terms
- COVID-19, SARS-CoV-2, Humans, Psychology, Social Media, Non-Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media. https://app.cochat.ai/papers/10.2196/22600
- The paper
- Saha, K., Torous, J., Caine, E. D., & De Choudhury, M. (2020). Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media. Journal of Medical Internet Research. https://doi.org/10.2196/22600
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.