Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media
Open accessPublished version of a preprint
The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index; 10 of the 71 sources its claims cite were checked for support.
Abstract
Background: The COVID-19 pandemic has caused several disruptions in personal and collective lives worldwide. The uncertainties surrounding the pandemic have also led to multifaceted mental health concerns, which can be exacerbated with precautionary measures such as social distancing and self-quarantining, as well as societal impacts such as economic downturn and job loss. Despite noting this as a “mental health tsunami”, the psychological effects of the COVID-19 crisis remain unexplored at scale. Consequently, public health stakeholders are currently limited in identifying ways to provide timely and tailored support during these circumstances. Objective: Our study aims to provide insights regarding people’s psychosocial concerns during the COVID-19 pandemic by leveraging social media data. We aim to study the temporal and linguistic changes in symptomatic mental health and support expressions in the pandemic context. Methods: We obtained about 60 million Twitter streaming posts originating from the United States from March 24 to May 24, 2020, and compared these with about 40 million posts from a comparable period in 2019 to attribute the effect of COVID-19 on people’s social media self-disclosure. Using these data sets, we studied people’s self-disclosure on social media in terms of symptomatic mental health concerns and expressions of support. We employed transfer learning classifiers that identified the social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support). We then examined the changes in psychosocial expressions over time and language, comparing the 2020 and 2019 data sets. Results: We found that all of the examined psychosocial expressions have significantly increased during the COVID-19 crisis—mental health symptomatic expressions have increased by about 14%, and support expressions have increased by about 5%, both thematically related to COVID-19. We also observed a steady decline and eventual plateauing in these expressions during the COVID-19 pandemic, which may have been due to habituation or due to supportive policy measures enacted during this period. Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis. Conclusions: We studied the psychosocial effects of the COVID-19 crisis by using social media data from 2020, finding that people’s mental health symptomatic and support expressions significantly increased during the COVID-19 period as compared to similar data from 2019. However, this effect gradually lessened over time, suggesting that people adapted to the circumstances and their “new normal.” Our linguistic analyses revealed that people expressed mental health concerns regarding personal and professional challenges, health care and precautionary measures, and pandemic-related awareness. This study shows the potential to provide insights to mental health care and stakeholders and policy makers in planning and implementing measures to mitigate mental health risks amid the health crisis.
Overview
CoChat Summary
TL;DR
This quasi-experimental Twitter study compared ~59 million US posts from March 24–May 24, 2020 with ~40.9 million posts from the same period in 2019, finding significant increases in symptomatic mental health and support expressions during COVID-19. The expressions were thematically tied to the pandemic and generally declined or plateaued over the two-month treatment period.
Key findings
- All examined psychosocial expressions increased significantly in 2020 versus 2019: symptomatic mental health expressions rose about 14% overall and support expressions about 5% overall.
- Specific increases were anxiety 21.32%, suicidal ideation 19.73%, depression 10.18%, stress 3.76%, emotional support 4.77%, and informational support 4.78%; all had significant t tests and Cohen d > 0.4.
- Temporal plots showed a negative slope in the treatment year (average slope = –0.03) versus a near-zero slope in control (average slope = 3.19×10⁻⁴), suggesting expressions leveled out over time.
- SAGE language analyses found treatment-period keywords such as covid19, coronavirus, social distancing, stayathome isolation, essential workers, dr fauci, masks, ppe, furloughed, and losing jobs, while control-period keywords included hospitality, trainee, crimes, traffic, and parttime.
Methods
- Quasi-experimental design using Twitter Streaming API data: treatment = 59,096,694 US English posts from March 24–May 24, 2020; control = 40,875,185 US English posts from March 24–May 24, 2019.
- Transfer-learning SVM classifiers (n-grams n=1,2,3) identified anxiety, depression, stress, suicidal ideation, emotional support, and informational support; classifiers were trained on Reddit data and reported ~0.90 average accuracy, 87% Twitter agreement with expert appraisal, and support classifier k-fold accuracies of 0.71 and 0.77.
- Analyses included two-sample t tests and Cohen d, z-score standardization using an additional year-long Twitter data set of over 240 million posts (September 2018–August 2019), SAGE for salient n-grams, and log-likelihood ratios for keyword overlap.
Limitations
- Stated: the Twitter Streaming API lacks transparency, the “1% Twitter stream” credibility has been questioned, and actual sampling may be smaller; the authors avoided regional comparisons and absolute-number claims, filtered English-only posts, and excluded data after May 24, 2020 to avoid confounding with George Floyd/Black Lives Matter events.
- Stated: Twitter data have self-selection and representation biases, the study is observational and does not measure true causality, and support expressions may include both support-seeking and support-providing.
- Evident from the text: mental health and support labels are machine-predicted proxies transferred from Reddit rather than clinical diagnoses, and findings are limited to one platform and a finite two-month period.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
8 Valid · 0 Not valid · 46 Not checked · 53 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Fear and anxiety about a disease as infectious as COVID-19 can trigger new-onset or exacerbate existing mental illness [8].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Our copy supports only the 'new-onset' half, and at a weaker, hedged level: the Abstract lists 'fear of contracting the disease' among the stressors COVID-19 introduced to mental health, and the Discussion (p. 1) says the Kessler 6 measure 'has been shown to accurately predict serious mental illness, suggesting acute distress during COVID-19 may transfer to longer-term psychiatric disorders.' Nothing in our copy addresses exacerbation of pre-existing mental illness, and the paper itself reports cross-sectional prevalence estimates (13.6% serious psychological distress in April 2020 vs 3.9% in 2018) rather than demonstrating that fear or anxiety triggers psychiatric disorders. The citing sentence also states as a general capability what the source frames as a suggestion ('may'). (LLM, conf=0.72)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Social media provides a real-time platform where people often candidly self-disclose their concerns, opinions, and struggles during this pandemic [23].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [23] sourceExperimental — accuracy not yet measured
The section “Utilization of Data to Understand Disparities” says people may voice opinions on social media that surveys do not capture, and that publicly shared posts may reveal mistrust or preferences. It does not establish that people often candidly disclose concerns and struggles during COVID-19, or describe social media specifically as a real-time platform. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
The Abstract identifies mental-health stressors beyond infection, and the Results report substantially more serious psychological distress among US adults in April 2020 than in 2018. The paper does not compare the crisis’s overall practical impact with the number of infections or deaths, so it does not establish the claimed ‘far greater’ magnitude. (LLM, conf=0.82)
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
The Introduction says the pandemic’s direct and indirect psychological and social effects are pervasive, and the section on lockdown and social isolation says the whole population is affected in different ways. This supports an impact beyond infections and deaths, but the paper does not establish the claim’s comparison that the practical impact is “far greater” than the number of cases or fatalities. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The observations reflected that the COVID-19 pandemic has increased people’s mental health expressions on social media, aligning with other contemporary literature and media reports [8,53].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Results > Temporal Variation
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Our copy of McGinty et al. (JAMA 2020) supports the general substance — the Discussion (Methods/Discussion section) states 'The prevalence of reported symptoms of psychological distress among US adults was higher in 2020 during the COVID-19 pandemic than in 2018' (13.6% vs 3.9% serious psychological distress, Results/Figure). However, the source measures distress and loneliness via the Johns Hopkins COVID-19 Civic Life and Public Health Survey on NORC's AmeriSpeak panel and the 2018 NHIS — administered online/in household interviews as structured survey instruments (Methods). No passage anywhere in our copy addresses social media, online posts, or 'mental health expressions' on platforms, so the specifically social-media component of the citing sentence is not attributable to this paper. (LLM, conf=0.70)
- Could not verifycites [53] sourceExperimental — accuracy not yet measured
Our copy of the Bedford et al. Lancet Comment is a STAG-IH policy piece on COVID-19 risk assessment and control measures (opening section, p.1; response-measure paragraphs and recommendations; research-gaps panel); none of these passages mention mental health, psychological distress, or social-media postings. Targeted searches ('mental health social media expressions' and 'psychological distress stigma online communication public awareness') returned only the same control/epidemiology passages plus the reference list, which cites WHO situation reports, the China CDC 72 314-case series, viral-load studies and social-distancing notices — no mental-health or social-media source. The paper therefore neither supports nor contradicts the attributed point; it simply does not address it. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis.”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 107 claims and to verify the 61 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 34
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Sample sizes ranged from two individuals in a number of case series ( Kapilan, 2020; Mamun et al., 2020b; Pirnia et al., 2020; Sahoo et al., 2020) to 60 million Twitter posts ( Saha et al., 2020).”
From The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review (2021) · cites this paper as Saha et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“The two studies we identified ( Low et al., 2020; Saha et al., 2020) reported heightened levels of suicide-related posting/suicidality.”
From The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review (2021) · cites this paper as Saha et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Methods > Examining Psychosocial Expressions Over Time and Language
“LLRs<1 indicated the greater frequency in symptomatic expressions, and LLRs>1 indicated the greater frequency in support expressions.”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Results > Linguistic Expressions
“dKeywords with LLRs<0 distinctly occurred in support expressions.”
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
The need to belong: Desire for interpersonal attachments as a fundamental human motivation.
Baumeister, R. F., & Leary, M. R. (1995) · Psychological Bulletin · doi:10.1037//0033-2909.117.3.497 · Cited by 23,245
Stress, social support, and the buffering hypothesis
Cohen, S., & Wills, T. A. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.98.2.310 · Cited by 18,093
Holmes, E. A., O'Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., Ballard, C., Christensen, H., Cohen Silver, R., Everall, I., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30168-1 · Cited by 5,715 · cited 1×
“Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9].”
Backgroundboyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479 · cited 1×
“However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-rel…”
MethodMental health and the COVID-19 pandemic
Pfefferbaum, B., & North, C. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2008017 · Cited by 4,378 · cited 2×
“Despite concerns about the myriad of social and behavioral issues associated with the COVID-19 pandemic [15,16], there has been scant research to examine its psychosocial impacts or how to predict and mitigate them.”
BackgroundPappa, S., Ntella, V., Giannakas, T., Giannakoulis, V. G., Papoutsi, E., & Katsaounou, P. (2020) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2020.05.026 · Cited by 3,617 · cited 1×
“The published literature posits that the distress and anxiety among individuals in this COVID-19 pandemic may increase the incidence of mental disorders [53,54,99]; data thus far from the United States point to a population increase in psychological distress of 10% compared to 2018 data [8], a trend that is in line wi…”
ResultMental health problems and social media exposure during COVID-19 outbreak
Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231924 · Cited by 2,830
Mental Health Problems and Social Media Exposure During COVID-19 Outbreak
Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3541120 · Cited by 2,830
Research works printed but not linked: 5
- SadilekAKautzHSilenzioVPredicting disease transmission from geo-tagged micro-blog data2012Twenty-Sixth AAAI Conference on Artificial IntelligenceJuly 2012Toronto, ON
- CoppersmithGAHarmanCTDredzeMHMeasuring post traumatic stress disorder in Twitter2014Eighth International AAAI Conference on Weblogs and Social Media2014Ann Arbor, MI
- Saha, K, 2018, Proc Int AAAI Conf Weblogs Soc Media, 2018, 320
- EisensteinJAhmedAXingEPSparse additive generative models of text201128th International Conference on International Conference on Machine LearningJune 2011Bellevue, WA
- Aassve, A, 2020, Working Papers 661, Innocenzo Gasparini Institute for Economic Research, 1
Cited by
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Self-citationZhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581318 · Cited by 288
Social media use in disaster recovery: A systematic literature review
Ogie, R. I., James, S., Moore, A., Dilworth, T., Amirghasemi, M., & Whittaker, J. (2022) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2022.102783 · Cited by 169
John, A., Eyles, E., Webb, R. T., Okolie, C., Schmidt, L., Arensman, E., Hawton, K., O'Connor, R. C., Kapur, N., Moran, P., et al. (2021) · F1000Research · doi:10.12688/f1000research.25522.2 · Cited by 189
The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: a living systematic review
John, A., Okolie, C., Eyles, E., Webb, R. T., Schmidt, L., McGuinness, L. A., Olorisade, B. K., Arensman, E., Hawton, K., Kapur, N., et al. (2020) · F1000Research · doi:10.12688/f1000research.25522.1 · Cited by 178
Yue, Z., Zhang, R., & Xiao, J. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107050 · Cited by 139
Luo, C., Chen, A., Cui, B., & Liao, W. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101712 · Cited by 161
Zheng, Y., Wang, W., Zhong, Y., Wu, F., Zhu, Z., Tham, Y.-C., Lamoureux, E., Xiao, L., Zhu, E., Liu, H., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/24316 · Cited by 73
What is Spiritual Support and How Might It Impact the Design of Online Communities?
Smith, C. E., Kaur, A., Gach, K. Z., Terveen, L., Kreitzer, M. J., & O’Conner-Von, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449117 · Cited by 83
“Some early studies are beginning to demonstrate shifts in OHC and social media usage as a result (e.g., [12, 61]).”
BackgroundGuntuku, S. C., Buttenheim, A. M., Sherman, G. T., & Merchant, R. M. (2021) · Vaccine · doi:10.1016/j.vaccine.2021.06.014 · Cited by 68
Gianfredi, V., Provenzano, S., & Santangelo, O. E. (2021) · Public Health · doi:10.1016/j.puhe.2021.06.024 · Cited by 68
Social Media Discussions Predict Mental Health Consultations on College Campuses
Saha, K., Yousuf, A., Boyd, R. L., Pennebaker, J. W., & Choudhury, M. D. (2022) · Scientific Reports · doi:10.1038/s41598-021-03423-4 · Cited by 91
Self-citationCollective emotions during the COVID-19 outbreak.
Metzler, H., Rimé, B., Pellert, M., Niederkrotenthaler, T., Natale, A. D., & García, D. (2022) · Emotion · doi:10.1037/emo0001111 · Cited by 65
Understanding Side Effects of Antidepressants: Large-scale Longitudinal Study on Social Media Data
Saha, K., Torous, J. B., Kıcıman, E., & Choudhury, M. D. (2021) · JMIR Mental Health · doi:10.2196/26589 · Cited by 60
Self-citationGu, D., Li, M., Yang, X., Gu, Y., Zhao, Y., Liang, C., & Hu, L. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103192 · Cited by 59
The role of social media in public health crises caused by infectious disease: a scoping review
Terry, K., Yang, F. H., Yao, Q., & Liu, C. (2023) · BMJ Global Health · doi:10.1136/bmjgh-2023-013515 · Cited by 61
Artificial intelligence for COVID-19: battling the pandemic with computational intelligence
Xu, Z., Su, C., Xiao, Y., & Wang, F. (2021) · Intelligent Medicine · doi:10.1016/j.imed.2021.09.001 · Cited by 53
Zhang, S., Liu, M., Li, Y., & Chung, J. E. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph181910079 · Cited by 52
Tan, H., Peng, S.-L., Zhu, C.-P., you, Z., Miao, M.-C., & Kuai, S. (2021) · Journal of Medical Internet Research · doi:10.2196/29150 · Cited by 43
“…showed the highest concerns and negative emotions to outbreak-related topics, and there was a gradual decrease in such negative emotions thereafter, as reported by Lwin et al [17], Su et al [11], Gupta et al [18], Saha et al [19], Xue et al [20], Wang et al [21], and Yu et al [22] (Figure 1).”
Mittal, S., & Choudhury, M. D. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580834 · Cited by 46
Mental Health Coping Stories on Social Media: A Causal-Inference Study of Papageno Effect
Yuan, Y., Saha, K., Keller, B., Isometsä, E. T., & Aledavood, T. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583350 · Cited by 57
Self-citationYalçın, M., Baş, A., Bilici, R., Özdemir, Y. Ö., Beştepe, E. E., Kurnaz, S., Güneş, M., Kurt, V., & Koyu, E. (2021) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-021-02171-0 · Cited by 36
Boonyarat, P., Liew, D.-J., & Chang, Y. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103706 · Cited by 36
Population-scale dietary interests during the COVID-19 pandemic
Gligorić, K., Chioléro, A., Kıcıman, E., White, R. W., & West, R. (2022) · Nature Communications · doi:10.1038/s41467-022-28498-z · Cited by 37
Laukkala, T., Suvisaari, J., Rosenström, T. H., Pukkala, E., Junttila, K., Haravuori, H., Tuisku, K., Haapa, T., & Jylhä, P. J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18052524 · Cited by 38
Supporters First: Understanding Online Social Support on Mental Health from a Supporter Perspective
Kim, M., Saha, K., Choudhury, M. D., & Choi, D. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579525 · Cited by 51
Self-citationTaylor, S. C., Smernoff, Z., Rajan, M., Steeman, S. L., Gehringer, B. N., Dow, H. C., Barzilay, R., Rader, D. J., Bućan, M., Almasy, L. A., et al. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.01.046 · Cited by 30
Hübner, T., Wolfgang, T., Theis, A.-C., Steber, M., Wiedenmann, L., Wöckel, A., Diessner, J., Hein, G., Gründahl, M., Kämmerer, U., et al. (2022) · Reproductive Health · doi:10.1186/s12978-022-01493-9 · Cited by 35
Detecting Mental Distresses Using Social Behavior Analysis in the Context of COVID-19: A Survey
Dhelim, S., Chen, L., Das, S. K., Ning, H., Nugent, C., Leavey, G., Pesch, D., White, E. B., & Burns, D. (2023) · ACM Computing Surveys · doi:10.1145/3589784 · Cited by 34
Cardiovascular Complications of COVID-19 Vaccines
Liu, R., Pan, J., Zhang, C., & Sun, X. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.840929 · Cited by 32
Resnik, P., Choudhury, M. D., Schafer, K. M., & Coppersmith, G. (2021) · Journal of Medical Internet Research · doi:10.2196/28990 · Cited by 28
Yuan, Y., Zhang, J., Aledavood, T., Zhang, R., & Saha, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790558 · Cited by 26
“The models demonstrate strong predictive performance, achieving an average accuracy of approximately 0.90 on held-out test sets, and have been validated in other studies [94, 96].”
InfluentialSelf-citationA Computational Linguistic Approach to Study Border Theory at Scale
Douglas, T., Capra, L., & Musolesi, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3640998 · Cited by 28
“This tracks with prior studies that have found social media posts during lockdown contain a higher frequency of mental health expressions than a comparable dataset from the same period in the previous year [33].”
ResultHaucke, M. N., Liu, S., & Heinzel, S. (2021) · JMIR Mental Health · doi:10.2196/29419 · Cited by 29
Matthes, J., Koban, K., Neureiter, A., & Stević, A. (2021) · Journal of Medical Internet Research · doi:10.2196/28700 · Cited by 31
“Large-scale analyses of people’s public social media activities exhibited more verbal expressions that would typically be considered as symptomatic of mental health hazards (eg, anxiety, depression, stress, and suicidal ideation), and outreaches for emotional and informational support [20].”
Online Self-Disclosure, Social Support, and User Engagement During the COVID-19 Pandemic
Lee, J., Rajtmajer, S. M., Srivatsavaya, E., & Wilson, S. (2023) · ACM Transactions on Social Computing · doi:10.1145/3617654 · Cited by 33
Kimani-Murage, E. W., Osogo, D., Nyamasege, C. K., Igonya, E. K., Ngira, D. O., & Harrington, J. A. (2022) · BMC Public Health · doi:10.1186/s12889-022-13638-3 · Cited by 28
Emotional-based pedagogy and facilitating EFL learners' perceived flow in online education
Abdolrezapour, P., & Ghanbari, N. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.960287 · Cited by 26
“The literature investigating learners’ psychological factors during this lockdown has focused on such factors as learners’ mental health (Patsali et al., 2020; Saha et al., 2020; Shigemura et al., 2020), learners’ attitude (Unger and Meiran, 2020), emotion regulation (Restubog et al.”
BackgroundSeruwagi, G. K., Nakidde, C., Lugada, E., Ssematiko, M., Ddamulira, D. P., Masaba, A., Luswata, B., Ochen, E. A., Okot, B. J., Muhangi, D., et al. (2022) · Conflict and Health · doi:10.1186/s13031-022-00451-3 · Cited by 25
Exploring Social Media for Early Detection of Depression in COVID-19 Patients
Wu, J., Wu, X., Hua, Y., Lin, S., Zheng, Y., & Yang, J. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583867 · Cited by 29
Predicting user emotional tone in mental disorder online communities
Silveira, B., Assumpção, H. S., Murai, F., & da Silva, A. P. C. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.07.014 · Cited by 20
Suicidal Behavior in the First Wave of the COVID-19 Pandemic
Giner, L., Vera-Varela, C., Vega, D., Zelada, G. M., & Guija, J. A. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01312-9 · Cited by 24
Tušl, M., Thelen, A., Marcus, K., Peters, A., Shalaeva, E. V., Scheckel, B., Sykora, M. D., Elayan, S., Naslund, J. A., Shankardass, K., et al. (2022) · Discover Mental Health · doi:10.1007/s44192-022-00017-y · Cited by 19
Kanaparthi, S. D., Patle, A., & Naik, K. J. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15316-x · Cited by 17
Yahya, N. H., & Rahim, H. A. (2023) · Language and Health · doi:10.1016/j.laheal.2023.10.001 · Cited by 19
Exploring the Social Trend Indications of Utilizing E-Commerce during and after COVID-19’s Hit
Fakieh, B., & Happonen, A. (2022) · Behavioral Sciences · doi:10.3390/bs13010005 · Cited by 17
“Domains like, but are not limited to, local and international government policies [22,23], the economy [24–26], tourism [24,27], nature and the environment [28,29], society, psychological [30] and social behavior [31–33], health [34,35], and impact on food security [36,37].”
On the State of NLP Approaches to Modeling Depression in Social Media: A Post-COVID-19 Outlook
Bucur, A.-M., Moldovan, A.-C., Parvatikar, K., Zampieri, M., KhudaBukhsh, A. R., & Dinu, L. P. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3540507 · Cited by 27
“Saha et al. (2020) [81] used the Sparse Additive Generative Model (SAGE) [95] to identify language indicative of mental disorders and compared the 2020 period to 2019 to monitor for changes in mental health at the beginning of the COVID-19 pandemic in the US.”
Ntompras, C., Drosatos, G., & Kaldoudi, E. (2021) · Journal of Computational Social Science · doi:10.1007/s42001-021-00150-8 · Cited by 16
“…in a set of 7301 global tweets [12], non-negative matrix factorization on COVID-19-related hate tweets in Arabic language [13], and a study that used transfer learning classifiers to analyze 60 million tweets originating from the USA to identify psychological effects of the pandemic [14].”
BackgroundZhang, S., Liu, Y., Song, S., Peng, S., & Xiong, M. (2022) · International Journal of General Medicine · doi:10.2147/ijgm.s347439 · Cited by 18
Rosenström, T. H., Tuisku, K., Suvisaari, J., Pukkala, E., Junttila, K., Haravuori, H., Elovainio, M. J., Haapa, T., Jylhä, P. J., & Laukkala, T. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04389-x · Cited by 18
Fluctuations of psychological states on Twitter before and during COVID-19
Massell, J., Lieb, R., Meyer, A. H., & Mayor, E. (2022) · PLoS ONE · doi:10.1371/journal.pone.0278018 · Cited by 16
Karagiannidis, A. G., Theodorakopoulou, M. P., Ferro, C. J., Ortíz, A., Soler, M. J., Halimi, J., Januszewicz, A., Persu, A., Kreutz, R. H., & Sarafidis, P. A. (2022) · Clinical Kidney Journal · doi:10.1093/ckj/sfac235 · Cited by 15
The COVID-19 pandemic, psychologists’ professional quality of life and mental health
Kercher, A., Rahman, J., & Pedersen, M. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1339869 · Cited by 16
“Estimates were that between 20 and 67% of psychologists experienced symptoms of burnout (Morse et al., 2012; McCormack et al., 2018; O’Connor et al., 2018; Simpson et al., 2019).”
BackgroundBuško, V., & Bezinović, P. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.647991 · Cited by 22
Social Networking Security during COVID‐19: A Systematic Literature Review
Abid, R., Rizwan, M., Veselý, P., Basharat, A., Tariq, U., & Javed, A. R. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/2975033 · Cited by 15
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Liang, P. P., Liu, T., Cai, A., Muszyński, M., Ishii, R., Allen, N., Auerbach, R. P., Brent, D. A., Salakhutdinov, R., & Morency, L. (2021) · doi:10.18653/v1/2021.acl-long.322 · Cited by 19
“This problem is particularly exacerbated as an “echo pandemic” of mental health problems have arisen in the wake of the COVID-19 pandemic (Inkster et al., 2021; Saha et al., 2020).”
BackgroundSix Feet Apart: Online Payments During the COVID-19 Pandemic
Shaikh, O., Ung, C., Yang, D., & Chacon, F. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555218 · Cited by 17
Yoo, N., & Jang, S. H. (2023) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2023.102119 · Cited by 14
“Due to the COVID-19 pandemic, many families have experienced financial or economic hardships (Kim, 2022, 2021; Sampson et al., 2021; Zajacova et al., 2020), such as difficulties in managing rent and mort-gage payments or job loss (Gratz et al., 2020; Matsubayashi et al., 2022; Saha et al., 2020;…”
McChesney, J. E., & Thompson, L. F. (2023) · Journal of Business and Psychology · doi:10.1007/s10869-023-09907-6 · Cited by 14
“Disclosures of mental health problems on social media, including LinkedIn, are becoming common (De Choudhury et al., 2017; Saha et al., 2020).”
Digital Inequality Through the Lens of Self-Disclosure
Lee, J., Rajtmajer, S. M., Srivatsavaya, E., & Wilson, S. (2021) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2021-0052 · Cited by 14
Constant, A., Conserve, D. F., Gallopel‐Morvan, K., & Raude, J. (2022) · JMIRx Med · doi:10.2196/32859 · Cited by 11
“The ensuing reductions in social (collective training sessions or sport events) and physical (barred access to exercise facilities or parks) opportunities to exercise had a direct negative effect on health behaviors and well-being [7-11].”
Gülnar, B., & Acar, N. (2021) · Nitel Sosyal Bilimler · doi:10.47105/nsb.904351 · Cited by 11
Han, B., Song, S., Liu, D., & Mo, J. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103924 · Cited by 11
Epidemic dreams: dreaming about health during the COVID-19 pandemic
Šćepanović, S., Aiello, L. M., Barrett, D., & Quercia, D. (2022) · Royal Society Open Science · doi:10.1098/rsos.211080 · Cited by 11
Takano, M., & Yokotani, K. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19351 · Cited by 11
“Consequently, the importance of social resources that are not affected by behavioral restrictions due to COVID-19, such as online social support, has increased (Gabbiadini et al. 2020; Saha et al. 2020).”
Background“I stand up for us”: Muslims’ feelings of stigmatization in response to terrorism on social media
Kaskeleviciute, R., Knupfer, H., & Matthes, J. (2024) · New Media & Society · doi:10.1177/14614448241236712 · Cited by 12
“…crisis, for instance, a large analysis of social media posts in the context of the COVID-19 pandemic identified “social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support)” (Saha et al., 2020: 1).”
Analyzing Psychotherapy on Twitter: An 11-Year Analysis of Tweets From Major U.S. Media Outlets
Álvarez-Mon, M. Á., Fernandez-Lazaro, C. I., Ortega, M. A., Vidal, C., Molina-Ruiz, R. M., Álvarez‐Mon, M., & Martínez‐González, M. Á. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.871113 · Cited by 12
Harrigian, K., & Dredze, M. H. (2022) · ACM Web Science Conference (WebSci) · doi:10.1145/3501247.3531566 · Cited by 11
Depression Detection on COVID 19 Tweets Using Chimp Optimization Algorithm
Meena, R., & Bai, V. T. (2022) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2022.025305 · Cited by 9
Rochford, B., Pendse, S. R., Kumar, N., & Choudhury, M. D. (2022) · JMIR Mental Health · doi:10.2196/43253 · Cited by 9
“Researchers attribute this plateauing effect to habituation to a “new normal,” given the protracted nature of the pandemic [61].”
InfluentialPhipps, J. E., Whipps, M. D. M., D’Souza, I. A., LaSalle, J. M., & Simmons, L. A. (2023) · Maternal and Child Health Journal · doi:10.1007/s10995-023-03657-w · Cited by 10
Kaliappan, S., Liu, C., Jain, Y., Karkar, R., & Saha, K. (2025) · JMIR Aging · doi:10.2196/68890 · Cited by 15
Self-citationWeger, R., Lossio-Ventura, J. A., Rose-McCandlish, M., Shaw, J. S., Sinclair, S., Pereira, F., Chung, J. Y., & Atlas, L. Y. (2022) · JMIR Mental Health · doi:10.2196/40899 · Cited by 10
Innovative public strategies in response to COVID‐19: A review of practices from China
Wu, Y., Cao, Z., Yang, J. H., Bi, X., Xiong, W., Feng, X., Yan, Y., Zhang, Z., & Zhang, Z. (2024) · Health care science · doi:10.1002/hcs2.122 · Cited by 8
Li, M., Gu, D., Li, R., Gu, Y., Hu, L., Su, K., Wang, X., & Zhang, G. (2025) · Journal of Medical Internet Research · doi:10.2196/60292 · Cited by 10
“Saha et al [23] explored the effects of topic consistency, language style adaptation, and respondent credibility of support postings on TalkLife on effective psychosocial outcomes.”
Jony, S. S. R., Haque, U., Webb, N. J., Spence, E. E., Rahman, M. S., Aghamohammadi, N., Lie, Y., Ángulo-Molina, A., Ananth, S., Ren, X., et al. (2021) · Behavioral Sciences · doi:10.3390/bs11080106 · Cited by 8
Kaya, N., & Güler, H. (2023) · Modern Care Journal · doi:10.5812/modernc-131139 · Cited by 8
Feldhege, J., Wolf, M., Moessner, M., & Bauer, S. (2022) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-022-02067-7 · Cited by 11
Immune Response in COVID-19-associated Acute Kidney Injury and Maladaptive Kidney Repair
Fu, Y., & Dong, Z. (2023) · Integrative Medicine in Nephrology and Andrology · doi:10.1097/imna-d-22-00022 · Cited by 8
“…Cp40 [93] Vitamin D Anti-inflammatory, activates macrophages Vitamin D3 [94–96] Psychosocial support Maintaining immunity Psychological guidance [97–99] AKI: Acute kidney injury, COVID-19: coronavirus disease 2019, SARS-CoV-2: syndrome coronavirus 2, TLR: Toll-like receptors, NET: neutrophil…”
Narratives of Anxiety and Depression on YouTube: A Corpus-Based Thematic Analysis
Huang, X.-X., Gan, Y.-C., & Syed, A. (2024) · Language and Health · doi:10.1016/j.laheal.2024.09.001 · Cited by 6
Quar, T. K., Rashid, M. F. N., Rosdi, M. Y. M., Ishak, W. S., & Chong, F. Y. (2022) · American Journal of Audiology · doi:10.1044/2022_aja-22-00033 · Cited by 7
Baumann, S., Kameg, B. N., Wiltrout, C. T., Murdoch, D., Pelcher, L., & Burke, J. (2022) · Health Promotion Practice · doi:10.1177/15248399221141688 · Cited by 7
“While there has been increased awareness among the general public regarding the importance of maintaining mental health during COVID-19 (Saha et al., 2020), a nuanced understanding of mental health stressors and supports among youth is critical for public health practice.”
BackgroundSentiments about Mental Health on Twitter—Before and during the COVID-19 Pandemic
Beierle, F., Pryss, R. C., & Aizawa, A. (2023) · Healthcare · doi:10.3390/healthcare11212893 · Cited by 7
“Saha et al. collected 60 million tweets from a two-month time window between March and May 2020, as well as 40 million tweets from the same time window in 2019 [29].”
BackgroundLin, S., Zhang, J., Wang, L., & Wang, S. (2024) · Journal of the Knowledge Economy · doi:10.1007/s13132-024-01805-3 · Cited by 8
“Young white-collar workers in professions with lesser demands for social interaction often prioritize information security and employ social media for shorter durations and with lower frequency (Saha et al., 2020).”
RetractedExploring English as a Foreign Language (EFL) Teachers’ Experience of Flow During Online Classes
Khajavi, Y., & Abdolrezapour, P. (2022) · Open Praxis · doi:10.55982/openpraxis.14.3.495 · Cited by 7
Swain, V. D., Ye, J., Ramesh, S. K., Mondal, A., Abowd, G. D., & Choudhury, M. D. (2024) · JMIR Formative Research · doi:10.2196/52316 · Cited by 7
“This approach is inspired by other studies that have used both public and private social media data to predict mental states [17,31].”
BackgroundResultPsychosocial impact of COVID-19 among adults in the southeastern United States
Richmond, J., Sanderson, M., Shrubsole, M. J., Holowatyj, A. N., Schlundt, D. G., & Aldrich, M. C. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107191 · Cited by 6
“Evidence in the U.S. is accumulating to suggest high rates of stress, depression, and anxiety symptoms since the start of the pandemic (Holingue et al., 2020; McGinty et al., 2020; Park et al., 2020; Saha et al., 2020; Xiong et al., 2020).”
Demotivation Of AFL Students In The Online Learning
Muassomah, M. (2023) · Ijaz Arabi Journal of Arabic Learning · doi:10.18860/ijazarabi.v6i1.16884 · Cited by 6
“Also, there is a tsunami of mental health among societies. including students who are facing a new culture in the process of learning languages (Saha et al., 2020).”
BackgroundVranken, I., Schreurs, L., Noon, E. J., & Vandenbosch, L. (2023) · Cyberpsychology Journal of Psychosocial Research on Cyberspace · doi:10.5817/cp2023-1-5 · Cited by 8
Minority Stress Experienced by LGBTQ Online Communities during the COVID-19 Pandemic
Yuan, Y., Verma, G., Keller, B., & Aledavood, T. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22201 · Cited by 10
“Recent studies on the effects of the COVID-19 pandemic show a considerable increase in psychological expressions, including expressions related to mental health issues and support, among the general population (Saha et al. 2020; Gallagher et al. 2020).”
BackgroundBertran, F. A., Kim, S., Chang, M., Dagan, E., Duval, J., Isbister, K., & Vidal, L. T. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3441311 · Cited by 5
Hayes, C., Wood, L. J., Gaden, N., Gennaro, S., Gross, A., Hudson-Jinks, T. M., Loescher, C. W., Maurer, M., Mittelman, M., Pearson, M. M., et al. (2021) · Nursing Administration Quarterly · doi:10.1097/naq.0000000000000475 · Cited by 5
Future behaviours decision-making regarding travel avoidance during COVID-19 outbreaks
Ito, K., Kanemitsu, S., Kimura, R., & Omori, R. (2022) · Scientific Reports · doi:10.1038/s41598-022-24323-1 · Cited by 5
Impacto da Pandemia na Discussão sobre Saúde Mental: O Caso do Discord no Brasil
Bento, P., Buzelin, A., Aquino, Y., Carvalho, I., Dutenhefner, P. R., Dayrell, L., Santana, C., Estanislau, V., Pappa, G. L., de Miranda, D. M., et al. (2024) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia.2024.243238 · Cited by 7
Kyröläinen, A.-J., & Kuperman, V. (2023) · Experimental Aging Research · doi:10.1080/0361073x.2023.2219188 · Cited by 6
“…a similarly favorable, and statistically identical emotional state of older adults, associated with higher levels of positive emotions and lower levels of negative emotions, for a similar report, see Saha, Torous, Caine, and De Choudhury (2020) and Naseem, Razzak, Khushi, Eklund, and Kim (2021).”
BackgroundInfluentialDubar, R. T., Greene, D., & Watkins, N. K. (2024) · Emerging Adulthood · doi:10.1177/21676968241291275 · Cited by 4
“Importantly, there is some evidence that the COVID-19 pandemic may have shifted social media norms, fi nding it more commonplace to share highly emotional, and negative or challenging personal experiences online (Saha et al., 2020).”
Sanmartin-Mendoza, P., Palacio, C., Coronel, B., Villegas, L., Tello, E. A., & Viloria-Núñez, C. (2025) · IEEE Engineering Management Review · doi:10.1109/emr.2025.3575426 · Cited by 4
“…methodological tool that allows for a more in-depth understanding of the emotional climate within organizations,addressing a gap in the literature where the collective impact of mental health,digital engagement, and pandemic-induced stress on performance has not been sufficiently explored [12].”
InfluentialZhou, J., Verma, G., Zhang, L., Chang, N., & Choudhury, M. D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710990 · Cited by 8
“This is especially true during crisis times when people are emotionally vulnerable [67, 71].”
Investigating COVID-19 Vaccine Messaging in Online Social Networks using Artificial Intelligence
Prabagar, K., Srikandabala, K., Loganathan, N., Silva, D. D., Gamage, G., Rathnayaka, P., Perera, A. S., & Alahakoon, D. (2022) · Conference on Human System Interactions · doi:10.1109/hsi55341.2022.9869484 · Cited by 6
“Koustov et al. [29] used tweets to analyse the psychosocial effects of the pandemic.”
BackgroundThe Increased Reliance on Social Media During COVID-19 Affects Individuals’ Mental Well-being
Bi, R. (2023) · Journal of Education Humanities and Social Sciences · doi:10.54097/ehss.v8i.4329 · Cited by 4
Metadata
- Authors
- Saha, K., Torous, J., Caine, E. D., & De Choudhury, M.
- Affiliations
- Georgia Institute of Technology (US)
- Beth Israel Deaconess Medical Center (US)
- University of Rochester Medicine (US)
- Corresponding author
- Saha, Koustuv
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-11-24
- Volume
- 22
- Issue
- 11
- Pages
- e22600
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7690250
- OpenAlex
- W3097203256
- Semantic Scholar
- b314968db7715d78c9bc4c73972f8311d477728a
- Cited by
- 166
- References
- 95
- Citations by year
- 2026: 11 · 2025: 20 · 2024: 20 · 2023: 30 · 2022: 57 · 2021: 28
- FWCI
- 31.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (2027689)
- National Institute of Mental Health (R01MH117172)
- National Institute of General Medical Sciences (R01GM112697)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › COVID-19 and Mental Health
- Social Sciences › Communication › Communication and COVID-19 Impact
- Keywords
- COVID-19 pandemic, mental health, social media self-disclosure, Twitter, anxiety, depression, stress, suicidal ideation, emotional support, information support, temporal trends, linguistic analysis, habituation, social distancing, public health policy, United States
- MeSH terms
- COVID-19, SARS-CoV-2, Humans, Psychology, Social Media, Non-Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.