Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media
Open accessPublished version of a preprint
The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index; 10 of the 71 sources its claims cite were checked for support.
Abstract
Background: The COVID-19 pandemic has caused several disruptions in personal and collective lives worldwide. The uncertainties surrounding the pandemic have also led to multifaceted mental health concerns, which can be exacerbated with precautionary measures such as social distancing and self-quarantining, as well as societal impacts such as economic downturn and job loss. Despite noting this as a “mental health tsunami”, the psychological effects of the COVID-19 crisis remain unexplored at scale. Consequently, public health stakeholders are currently limited in identifying ways to provide timely and tailored support during these circumstances. Objective: Our study aims to provide insights regarding people’s psychosocial concerns during the COVID-19 pandemic by leveraging social media data. We aim to study the temporal and linguistic changes in symptomatic mental health and support expressions in the pandemic context. Methods: We obtained about 60 million Twitter streaming posts originating from the United States from March 24 to May 24, 2020, and compared these with about 40 million posts from a comparable period in 2019 to attribute the effect of COVID-19 on people’s social media self-disclosure. Using these data sets, we studied people’s self-disclosure on social media in terms of symptomatic mental health concerns and expressions of support. We employed transfer learning classifiers that identified the social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support). We then examined the changes in psychosocial expressions over time and language, comparing the 2020 and 2019 data sets. Results: We found that all of the examined psychosocial expressions have significantly increased during the COVID-19 crisis—mental health symptomatic expressions have increased by about 14%, and support expressions have increased by about 5%, both thematically related to COVID-19. We also observed a steady decline and eventual plateauing in these expressions during the COVID-19 pandemic, which may have been due to habituation or due to supportive policy measures enacted during this period. Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis. Conclusions: We studied the psychosocial effects of the COVID-19 crisis by using social media data from 2020, finding that people’s mental health symptomatic and support expressions significantly increased during the COVID-19 period as compared to similar data from 2019. However, this effect gradually lessened over time, suggesting that people adapted to the circumstances and their “new normal.” Our linguistic analyses revealed that people expressed mental health concerns regarding personal and professional challenges, health care and precautionary measures, and pandemic-related awareness. This study shows the potential to provide insights to mental health care and stakeholders and policy makers in planning and implementing measures to mitigate mental health risks amid the health crisis.
Overview
CoChat Summary
TL;DR
This quasi-experimental Twitter study compared ~59 million US posts from March 24–May 24, 2020 with ~40.9 million posts from the same period in 2019, finding significant increases in symptomatic mental health and support expressions during COVID-19. The expressions were thematically tied to the pandemic and generally declined or plateaued over the two-month treatment period.
Key findings
- All examined psychosocial expressions increased significantly in 2020 versus 2019: symptomatic mental health expressions rose about 14% overall and support expressions about 5% overall.
- Specific increases were anxiety 21.32%, suicidal ideation 19.73%, depression 10.18%, stress 3.76%, emotional support 4.77%, and informational support 4.78%; all had significant t tests and Cohen d > 0.4.
- Temporal plots showed a negative slope in the treatment year (average slope = –0.03) versus a near-zero slope in control (average slope = 3.19×10⁻⁴), suggesting expressions leveled out over time.
- SAGE language analyses found treatment-period keywords such as covid19, coronavirus, social distancing, stayathome isolation, essential workers, dr fauci, masks, ppe, furloughed, and losing jobs, while control-period keywords included hospitality, trainee, crimes, traffic, and parttime.
Methods
- Quasi-experimental design using Twitter Streaming API data: treatment = 59,096,694 US English posts from March 24–May 24, 2020; control = 40,875,185 US English posts from March 24–May 24, 2019.
- Transfer-learning SVM classifiers (n-grams n=1,2,3) identified anxiety, depression, stress, suicidal ideation, emotional support, and informational support; classifiers were trained on Reddit data and reported ~0.90 average accuracy, 87% Twitter agreement with expert appraisal, and support classifier k-fold accuracies of 0.71 and 0.77.
- Analyses included two-sample t tests and Cohen d, z-score standardization using an additional year-long Twitter data set of over 240 million posts (September 2018–August 2019), SAGE for salient n-grams, and log-likelihood ratios for keyword overlap.
Limitations
- Stated: the Twitter Streaming API lacks transparency, the “1% Twitter stream” credibility has been questioned, and actual sampling may be smaller; the authors avoided regional comparisons and absolute-number claims, filtered English-only posts, and excluded data after May 24, 2020 to avoid confounding with George Floyd/Black Lives Matter events.
- Stated: Twitter data have self-selection and representation biases, the study is observational and does not measure true causality, and support expressions may include both support-seeking and support-providing.
- Evident from the text: mental health and support labels are machine-predicted proxies transferred from Reddit rather than clinical diagnoses, and findings are limited to one platform and a finite two-month period.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
8 Valid · 0 Not valid · 46 Not checked · 53 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Fear and anxiety about a disease as infectious as COVID-19 can trigger new-onset or exacerbate existing mental illness [8].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Our copy supports only the 'new-onset' half, and at a weaker, hedged level: the Abstract lists 'fear of contracting the disease' among the stressors COVID-19 introduced to mental health, and the Discussion (p. 1) says the Kessler 6 measure 'has been shown to accurately predict serious mental illness, suggesting acute distress during COVID-19 may transfer to longer-term psychiatric disorders.' Nothing in our copy addresses exacerbation of pre-existing mental illness, and the paper itself reports cross-sectional prevalence estimates (13.6% serious psychological distress in April 2020 vs 3.9% in 2018) rather than demonstrating that fear or anxiety triggers psychiatric disorders. The citing sentence also states as a general capability what the source frames as a suggestion ('may'). (LLM, conf=0.72)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Social media provides a real-time platform where people often candidly self-disclose their concerns, opinions, and struggles during this pandemic [23].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [23] sourceExperimental — accuracy not yet measured
The section “Utilization of Data to Understand Disparities” says people may voice opinions on social media that surveys do not capture, and that publicly shared posts may reveal mistrust or preferences. It does not establish that people often candidly disclose concerns and struggles during COVID-19, or describe social media specifically as a real-time platform. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
The Abstract identifies mental-health stressors beyond infection, and the Results report substantially more serious psychological distress among US adults in April 2020 than in 2018. The paper does not compare the crisis’s overall practical impact with the number of infections or deaths, so it does not establish the claimed ‘far greater’ magnitude. (LLM, conf=0.82)
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
The Introduction says the pandemic’s direct and indirect psychological and social effects are pervasive, and the section on lockdown and social isolation says the whole population is affected in different ways. This supports an impact beyond infections and deaths, but the paper does not establish the claim’s comparison that the practical impact is “far greater” than the number of cases or fatalities. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The observations reflected that the COVID-19 pandemic has increased people’s mental health expressions on social media, aligning with other contemporary literature and media reports [8,53].”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Results > Temporal Variation
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Our copy of McGinty et al. (JAMA 2020) supports the general substance — the Discussion (Methods/Discussion section) states 'The prevalence of reported symptoms of psychological distress among US adults was higher in 2020 during the COVID-19 pandemic than in 2018' (13.6% vs 3.9% serious psychological distress, Results/Figure). However, the source measures distress and loneliness via the Johns Hopkins COVID-19 Civic Life and Public Health Survey on NORC's AmeriSpeak panel and the 2018 NHIS — administered online/in household interviews as structured survey instruments (Methods). No passage anywhere in our copy addresses social media, online posts, or 'mental health expressions' on platforms, so the specifically social-media component of the citing sentence is not attributable to this paper. (LLM, conf=0.70)
- Could not verifycites [53] sourceExperimental — accuracy not yet measured
Our copy of the Bedford et al. Lancet Comment is a STAG-IH policy piece on COVID-19 risk assessment and control measures (opening section, p.1; response-measure paragraphs and recommendations; research-gaps panel); none of these passages mention mental health, psychological distress, or social-media postings. Targeted searches ('mental health social media expressions' and 'psychological distress stigma online communication public awareness') returned only the same control/epidemiology passages plus the reference list, which cites WHO situation reports, the China CDC 72 314-case series, viral-load studies and social-distancing notices — no mental-health or social-media source. The paper therefore neither supports nor contradicts the attributed point; it simply does not address it. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis.”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 107 claims and to verify the 61 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 34
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Sample sizes ranged from two individuals in a number of case series ( Kapilan, 2020; Mamun et al., 2020b; Pirnia et al., 2020; Sahoo et al., 2020) to 60 million Twitter posts ( Saha et al., 2020).”
From The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review (2021) · cites this paper as Saha et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“The two studies we identified ( Low et al., 2020; Saha et al., 2020) reported heightened levels of suicide-related posting/suicidality.”
From The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review (2021) · cites this paper as Saha et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Methods > Examining Psychosocial Expressions Over Time and Language
“LLRs<1 indicated the greater frequency in symptomatic expressions, and LLRs>1 indicated the greater frequency in support expressions.”
§ Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media > Results > Linguistic Expressions
“dKeywords with LLRs<0 distinctly occurred in support expressions.”
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
The need to belong: Desire for interpersonal attachments as a fundamental human motivation.
Baumeister, R. F., & Leary, M. R. (1995) · Psychological Bulletin · doi:10.1037//0033-2909.117.3.497 · Cited by 23,245
Stress, social support, and the buffering hypothesis
Cohen, S., & Wills, T. A. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.98.2.310 · Cited by 18,093
Holmes, E. A., O'Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., Ballard, C., Christensen, H., Cohen Silver, R., Everall, I., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30168-1 · Cited by 5,715 · cited 1×
“Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9].”
Backgroundboyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479 · cited 1×
“However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-rel…”
MethodMental health and the COVID-19 pandemic
Pfefferbaum, B., & North, C. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2008017 · Cited by 4,378 · cited 2×
“Despite concerns about the myriad of social and behavioral issues associated with the COVID-19 pandemic [15,16], there has been scant research to examine its psychosocial impacts or how to predict and mitigate them.”
BackgroundPappa, S., Ntella, V., Giannakas, T., Giannakoulis, V. G., Papoutsi, E., & Katsaounou, P. (2020) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2020.05.026 · Cited by 3,617 · cited 1×
“The published literature posits that the distress and anxiety among individuals in this COVID-19 pandemic may increase the incidence of mental disorders [53,54,99]; data thus far from the United States point to a population increase in psychological distress of 10% compared to 2018 data [8], a trend that is in line wi…”
ResultMental health problems and social media exposure during COVID-19 outbreak
Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231924 · Cited by 2,830
Mental Health Problems and Social Media Exposure During COVID-19 Outbreak
Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3541120 · Cited by 2,830
Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts
Hellewell, J., Abbott, S., Gimma, A., Bosse, N. I., Jarvis, C. I., Russell, T. W., Munday, J. D., Kucharski, A. J., Edmunds, W. J., Sun, F., et al. (2020) · The Lancet Global Health · doi:10.1016/s2214-109x(20)30074-7 · Cited by 2,550
“Pandemic fear” and COVID-19: mental health burden and strategies
Ornell, F., Schuch, J. B., Sordi, A. O., & Kessler, F. H. P. (2020) · Brazilian Journal of Psychiatry · doi:10.1590/1516-4446-2020-0008 · Cited by 1,852
Rankin, C. H., Abrams, T., Barry, R. J., Bhatnagar, S., Clayton, D. F., Colombo, J., Coppola, G., Geyer, M. A., Glanzman, D. L., Marsland, S., et al. (2009) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2008.09.012 · Cited by 1,665 · cited 1×
“This could likely be due to people’s habituation with the situation and surroundings with the passage of time [70,72], as has been observed for other crisis events”
BackgroundCOVID-19: towards controlling of a pandemic
Bedford, J., Enria, D., Giesecke, J., Heymann, D. L., Ihekweazu, C., Kobinger, G., Lane, H. C., Memish, Z., Oh, M.-D., Sall, A. A., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30673-5 · Cited by 1,545
González-Sanguino, C., Ausín, B., Castellanos, M. Á., Saiz, J., López-Gómez, A., Ugidos, C., & Muñoz, M. (2020) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2020.05.040 · Cited by 1,373
Varatharaj, A., Thomas, N., Ellul, M. A., Davies, N. W. S., Pollak, T. A., Tenorio, E. L., Sultan, M., Easton, A., Breen, G., Zandi, M., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30287-x · Cited by 1,246 · cited 1×
“Although disease mitigating efforts such as “social distancing” and “self-quarantining” are recommended [10-12], individuals in medical isolation may experience increased symptoms of anxiety and depression, as well as feelings of fear, abandonment, loneliness, and stigmatization [13,14].”
BackgroundThe consequences of the COVID-19 pandemic on mental health and implications for clinical practice
Fiorillo, A., & Gorwood, P. (2020) · European Psychiatry · doi:10.1192/j.eurpsy.2020.35 · Cited by 1,190
Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures
Golder, S. A., & Macy, M. W. (2011) · Science · doi:10.1126/science.1202775 · Cited by 1,155 · cited 1×
“Similar standardization techniques have been adopted in prior social media time series studies [48,69].”
MethodCOVID-19: what is next for public health?
Heymann, D. L., & Shindo, N. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30374-3 · Cited by 1,138
Online communication, social media and adolescent wellbeing: A systematic narrative review
Best, P., Manktelow, R., & Taylor, B. (2014) · Children and Youth Services Review · doi:10.1016/j.childyouth.2014.03.001 · Cited by 1,067 · cited 1×
“In the current pandemic, the additional layer of extensive social media use and exposure to often sensationalized online news while in physical isolation may add new complexities to implementing emotional epidemiology in managing concerns, fears, and misconceptions [96,97], as these tools have been argued to bear nega…”
BackgroundSuicide mortality and coronavirus disease 2019—a perfect storm?
Reger, M. A., Stanley, I. H., & Joiner, T. E. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1060 · Cited by 1,026
Psychological distress and loneliness reported by US adults in 2018 and April 2020
McGinty, E. E., Presskreischer, R., Han, H., & Barry, C. L. (2020) · JAMA · doi:10.1001/jama.2020.9740 · Cited by 978 · cited 5×
“Fear and anxiety about a disease as infectious as COVID-19 can trigger new-onset or exacerbate existing mental illness [8].”
BackgroundResultInfluentialYou Are What You Tweet: Analyzing Twitter for Public Health
Paul, M., & Dredze, M. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14137 · Cited by 947 · cited 2×
“Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].”
BackgroundResultChloroquine and hydroxychloroquine as available weapons to fight COVID-19
Colson, P., Rolain, J.-M., Lagier, J.-C., Brouqui, P., & Raoult, D. (2020) · International Journal of Antimicrobial Agents · doi:10.1016/j.ijantimicag.2020.105932 · Cited by 937 · cited 1×
“Further, we found keywords such as essential workers, doctor jobs, and risking lives, which describe high-risk worker situations (eg, “I am not complaining about going to work, rather, I am concerned about risking my health for work”), and certain treatment suggestions that evolved during this period [76], such as gar…”
Controllability of Stressful Events and Satisfaction With Spouse Support Behaviors
CUTRONA, C. E., & SUHR, J. A. (1992) · Communication Research · doi:10.1177/009365092019002002 · Cited by 928 · cited 1×
“According to the “Social Support Behavioral Code” [64], two forms of support that have received theoretical and empirical attention are emotional and informational support.”
BackgroundMental Health Discourse on reddit: Self-Disclosure, Social Support, and Anonymity
De Choudhury, M., & De, S. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14526 · Cited by 836
Linguistic Markers of Psychological Change Surrounding September 11, 2001
Cohn, M. A., Mehl, M. R., & Pennebaker, J. W. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00741.x · Cited by 808
Processing Social Media Messages in Mass Emergency: A Survey
Imran, M., Castillo, C., Diaz, F., & Vieweg, S. (2015) · ACM Computing Surveys · doi:10.1145/2771588 · Cited by 804
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media
De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., & Kumar, M. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858207 · Cited by 683
Methods in predictive techniques for mental health status on social media: a critical review
Chancellor, S., & De Choudhury, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0233-7 · Cited by 557
Social media use and mental health among young adults
Berryman, C., Ferguson, C. J., & Negy, C. (2018) · Psychiatric Quarterly · doi:10.1007/s11126-017-9535-6 · Cited by 484 · cited 1×
“In the current pandemic, the additional layer of extensive social media use and exposure to often sensationalized online news while in physical isolation may add new complexities to implementing emotional epidemiology in managing concerns, fears, and misconceptions [96,97], as these tools have been argued to bear nega…”
BackgroundWorld Health Organization (2020) · Pediatria i Medycyna Rodzinna · doi:10.15557/pimr.2020.0004 · Cited by 468
Andalibi, N., Haimson, O. L., De Choudhury, M., & Forte, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858096 · Cited by 378 · cited 3×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundAttai, D. J., Cowher, M. S., Al-Hamadani, M., Schoger, J. M., Staley, A. C., & Landercasper, J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4721 · Cited by 357 · cited 1×
“Andalibi et al [66] found that these two kinds of support can co-occur with other forms of support, such as posts seeking ES often seeking esteem and network support, and Attai et al [68] noted that Twitter is effective in seeking and providing health-related informational needs, contextually related with our problem…”
BackgroundIntolerance of uncertainty, appraisals, coping, and anxiety: The case of the 2009 H 1 N 1 pandemic
Taha, S., Matheson, K., Cronin, T., & Anisman, H. (2014) · British Journal of Health Psychology · doi:10.1111/bjhp.12058 · Cited by 323 · cited 2×
“The published literature posits that the distress and anxiety among individuals in this COVID-19 pandemic may increase the incidence of mental disorders [53,54,99]; data thus far from the United States point to a population increase in psychological distress of 10% compared to 2018 data [8], a trend that is in line wi…”
BackgroundResultWang, Y.-C., Kraut, R., & Levine, J. M. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145329 · Cited by 282
Nambisan, P. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2010-000058 · Cited by 282 · cited 1×
“Emotional support (ES) corresponds to empathy, encouragement, and kindness, while informational support (IS) corresponds to information, guidance, and suggestions [38,65].”
BackgroundRains, S. A., & Young, V. (2009) · Human Communication Research · doi:10.1111/j.1468-2958.2009.01353.x · Cited by 265 · cited 1×
“In fact, a meta-analysis indicated that online support is effective in decreasing depression and increasing self-efficacy and quality of life [60].”
BackgroundOh, H. J., Lauckner, C., Boehmer, J., Fewins-Bliss, R., & Li, K. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.04.017 · Cited by 256 · cited 2×
“These two forms of support have been found to be most prevalent and effective in several studies of online support and social media [38,62,66,67].”
BackgroundEpidemics and Pandemics: Their Impacts on Human History
Hays, J. N. (2005) · ABC-CLIO eBooks · doi:10.5040/9798400647055 · Cited by 251
Social support in a wired world: Use of online mental health forums in Norway
Kummervold, P. E., Gammon, D., Bergvik, S., Johnsen, J.-A. K., Hasvold, T., & Rosenvinge, J. H. (2002) · Nordic Journal of Psychiatry · doi:10.1080/08039480252803945 · Cited by 223 · cited 2×
“Research reports that supportive interactions can even have a “buffering effect” [57,58]; that is, they can be protective against the negative consequences of mental health.”
BackgroundPutting Their Best Foot Forward: Emotional Disclosure on Facebook
Qiu, L., Lin, H., Leung, A. K., & Tov, W. (2012) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2012.0200 · Cited by 211 · cited 1×
“Over 80% of US adults use social media daily [20], placing it ahead of texting, email, and instant messaging, and disclose considerably more about themselves online than offline [21,22].”
BackgroundHow Social Media Will Change Public Health
Dredze, M. (2012) · IEEE Intelligent Systems · doi:10.1109/mis.2012.76 · Cited by 207 · cited 4×
“Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].”
BackgroundResultInfluentialThe Language of Social Support in Social Media and Its Effect on Suicidal Ideation Risk
De Choudhury, M., & Kiciman, E. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14891 · Cited by 202
Social Support, Reciprocity, and Anonymity in Responses to Sexual Abuse Disclosures on Social Media
Andalibi, N., Haimson, O. L., Choudhury, M. D., & Forte, A. (2018) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3234942 · Cited by 200 · cited 2×
“Andalibi et al [66] found that these two kinds of support can co-occur with other forms of support, such as posts seeking ES often seeking esteem and network support, and Attai et al [68] noted that Twitter is effective in seeking and providing health-related informational needs, contextually related with our problem…”
BackgroundJacobson, N. C., Lekkas, D., Price, G., Heinz, M. V., Song, M., O’Malley, A. J., & Barr, P. J. (2020) · doi:10.2196/preprints.19347 · Cited by 195
Jacobson, N. C., Lekkas, D., Price, G., Heinz, M. V., Song, M., O’Malley, A. J., & Barr, P. J. (2020) · JMIR Mental Health · doi:10.2196/19347 · Cited by 195
Common Sense and Sociological Explanations
Watts, D. J. (2014) · American Journal of Sociology · doi:10.1086/678271 · Cited by 176
Nelson, B. W., Pettitt, A., Flannery, J. E., & Allen, N. B. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241990 · Cited by 175
Coronavirus Disease 2019 (COVID-19)
Saxena, S. K. (2020) · Medical virology · doi:10.1007/978-981-15-4814-7 · Cited by 172
Mental Health Support and its Relationship to Linguistic Accommodation in Online Communities
Sharma, E., & De Choudhury, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174215 · Cited by 156 · cited 6×
“When the predictions of these classifiers were cross-validated with expert annotations from Sharma and De Choudhury’s [38] data, the classifiers were found to have k-fold cross-validation accuracies of 0.”
BackgroundMethodInfluentialEpidemics and trust: the case of the Spanish flu
Aassve, A., Alfani, G., Gandolfi, F., & Le Moglie, M. (2021) · Health Economics · doi:10.1002/hec.4218 · Cited by 156 · cited 1×
“The backdrop of the 1918 pandemic was that it happened just before the advent of modern psychiatry as a science and a clinical specialty—a time when psychoanalysis was gaining recognition as an established treatment within the medical community [91,92].”
BackgroundCognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load
Abdelrahman, Y., Velloso, E., Dingler, T., Schmidt, A., & Vetere, F. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130898 · Cited by 154
Influenza: historical aspects of epidemics and pandemics
Cunha, B. A. (2004) · Infectious Disease Clinics of North America · doi:10.1016/s0891-5520(03)00095-3 · Cited by 150 · cited 1×
“COVID-19 is not the first pandemic—catastrophic pandemics have been occurring at regular intervals throughout human history, with the 1918 influenza epidemic being the last one before the current pandemic [84,85,90].”
BackgroundGore, R. J., Diallo, S. Y., & Padilla, J. J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0133505 · Cited by 150
Mackey, T., Purushothaman, V., Li, J., Shah, N., Nali, M., Bardier, C., Liang, B., Cai, M., & Cuomo, R. (2020) · JMIR Public Health and Surveillance · doi:10.2196/19509 · Cited by 142
The Language of LGBTQ+ Minority Stress Experiences on Social Media
Saha, K., Kim, S. C., Reddy, M. D., Carter, A. J., Sharma, E., Haimson, O. L., & Choudhury, M. D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3361108 · Cited by 141 · cited 2×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundSelf-citationTracking Mental Health and Symptom Mentions on Twitter During COVID-19
Guntuku, S. C., Sherman, G. T., Stokes, D. C., Agarwal, A. K., Seltzer, E., Merchant, R. M., & Ungar, L. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05988-8 · Cited by 140
Tampering with Twitter’s Sample API
Pfeffer, J., Mayer, K., & Morstatter, F. (2018) · EPJ Data Science · doi:10.1140/epjds/s13688-018-0178-0 · Cited by 135 · cited 3×
“Recent research has also questioned the credibility of the “1% Twitter stream” aspect, noting that actual sampling data was smaller than what it ideally should have been [41].”
BackgroundMethodBurton, S. H., Tanner, K., Giraud-Carrier, C. G., West, J. H., & Barnes, M. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2121 · Cited by 127 · cited 1×
“However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-rel…”
MethodModeling Stress with Social Media Around Incidents of Gun Violence on College Campuses
Saha, K., & De Choudhury, M. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134727 · Cited by 124 · cited 6×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundMethodInfluentialSelf-citation"Zero suicide" - a model for reducing suicide in United States behavioral healthcare
Labouliere, C. D., Vasan, P., Kramer, A., Brown, G., Green, K., Kammer, J., Finnerty, M., & Stanley, B. (2018) · Suicidologi · doi:10.5617/suicidologi.6198 · Cited by 118
A Social Media Based Index of Mental Well-Being in College Campuses
Bagroy, S., Kumaraguru, P., & De Choudhury, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025909 · Cited by 115 · cited 3×
“We further manually inspected a random set of 125 Twitter posts in our data set using the methods outlined in prior studies [25,56] to rate each Twitter post with binary high or low ES and IS.”
BackgroundMethodDo Cancer Patients Tweet? Examining the Twitter Use of Cancer Patients in Japan
Tsuya, A., Sugawara, Y., Tanaka, A., & Narimatsu, H. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3298 · Cited by 109 · cited 1×
“However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-rel…”
MethodA Social Media Study on the Effects of Psychiatric Medication Use
Saha, K., Sugar, B., Torous, J., Abrahao, B., Kıcıman, E., & De Choudhury, M. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3242 · Cited by 96 · cited 9×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundMethodInfluentialSelf-citationA computational study of mental health awareness campaigns on social media
Saha, K., Torous, J., Ernala, S. K., Rizuto, C., Stafford, A., & De Choudhury, M. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz028 · Cited by 93
Self-citationWhen is it biased?: assessing the representativeness of twitter's streaming API
Morstatter, F., Pfeffer, J., & Liu, H. (2014) · Proceedings of the 23rd International Conference on World Wide Web · doi:10.1145/2567948.2576952 · Cited by 90
Language of ADHD in Adults on Social Media
Guntuku, S. C., Ramsay, J. R., Merchant, R. M., & Ungar, L. H. (2019) · Journal of Attention Disorders · doi:10.1177/1087054717738083 · Cited by 89 · cited 1×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundSocial distancing prevents infections, but it can have unintended consequences
Miller, G. (2020) · Science · doi:10.1126/science.abb7506 · Cited by 86 · cited 1×
“Although disease mitigating efforts such as “social distancing” and “self-quarantining” are recommended [10-12], individuals in medical isolation may experience increased symptoms of anxiety and depression, as well as feelings of fear, abandonment, loneliness, and stigmatization [13,14].”
BackgroundDigital Mental Health: The Answer to the Global Mental Health Crisis?
Rudd, B. N., & Beidas, R. S. (2020) · JMIR Mental Health · doi:10.2196/18472 · Cited by 81
Pendse, S. R., Sharma, A., Vashistha, A., Choudhury, M. D., & Kumar, N. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445410 · Cited by 80
Stokes, D. C., Andy, A., Guntuku, S. C., Ungar, L. H., & Merchant, R. M. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05889-w · Cited by 77
Inferring Mood Instability on Social Media by Leveraging Ecological Momentary Assessments
Saha, K., Chan, L., De Barbaro, K., Abowd, G. D., & De Choudhury, M. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130960 · Cited by 77 · cited 3×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundMethodSelf-citationPandemics: Health Care Emergencies
Morganstein, J. C., Fullerton, C. S., Ursano, R. J., Donato, D., & Holloway, H. C. (2017) · Textbook of Disaster Psychiatry · doi:10.1017/9781316481424.019 · Cited by 76
Processing Social Media Messages in Mass Emergency
Imran, M., Castillo, C., Díaz, F., & Vieweg, S. E. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3184558.3186242 · Cited by 76
Causal Factors of Effective Psychosocial Outcomes in Online Mental Health Communities
Saha, K., & Sharma, A. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7326 · Cited by 72
Self-citation"Narco" Emotions: Affect and Desensitization in Social Media during the Mexican Drug WarPreprint
Choudhury, M. D., Monroy‐Hernández, A., & Mark, G. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01287 · Cited by 67
"Narco" emotions: affect and desensitization in social media during the mexican drug war
De Choudhury, M., Monroy-Hernández, A., & Mark, G. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557197 · Cited by 62
Using Longitudinal Social Media Analysis to Understand the Effects of Early College Alcohol Use
Kıcıman, E., Counts, S., & Gasser, M. L. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15012 · Cited by 60
Saha, K., Weber, I., & De Choudhury, M. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15016 · Cited by 55 · cited 1×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundSelf-citationPendse, S. R., Lalani, F. M., De Choudhury, M., Sharma, A., & Kumar, N. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376465 · Cited by 53 · cited 2×
“Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].”
BackgroundResultAssessing Behavior Stage Progression From Social Media Data
Liu, J., Weitzman, E. R., & Chunara, R. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998336 · Cited by 52
Ernala, S. K., Labetoulle, T., Bane, F., Birnbaum, M. L., Rizvi, A. F., Kane, J. M., & De Choudhury, M. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15027 · Cited by 50 · cited 5×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundResultInfluentialSocial Media as a Passive Sensor in Longitudinal Studies of Human Behavior and Wellbeing
Saha, K., Bayraktaroglu, A. E., Campbell, A. T., Chawla, N. V., Choudhury, M. D., D’Mello, S. K., Dey, A. K., Gao, G., Gregg, J. M., Jagannath, K., et al. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299065 · Cited by 50 · cited 1×
“Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].”
ResultSelf-citationThe Recent Wave of 'Spanish' Flu Historiography
Phillips, H. (2014) · Social History of Medicine · doi:10.1093/shm/hku066 · Cited by 44 · cited 1×
“The backdrop of the 1918 pandemic was that it happened just before the advent of modern psychiatry as a science and a clinical specialty—a time when psychoanalysis was gaining recognition as an established treatment within the medical community [91,92].”
BackgroundLuo, C., Li, Y., Chen, A., & Tang, Y. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241465 · Cited by 44
Saha, K., Weber, I. G., Birnbaum, M. L., & Choudhury, M. D. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6815 · Cited by 41 · cited 2×
“Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].”
BackgroundResultSelf-citationVariability in Language used on Social Media prior to Hospital Visits
Guntuku, S. C., Schwartz, H. A., Kashyap, A., Gaulton, J. S., Stokes, D. C., Asch, D. A., Ungar, L., & Merchant, R. M. (2020) · Scientific Reports · doi:10.1038/s41598-020-60750-8 · Cited by 39 · cited 2×
“Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52].”
BackgroundYoo, D. W., Birnbaum, M. L., Meter, A. V., Ali, A. F., Arenare, E., Abowd, G. D., & Choudhury, M. D. (2020) · JMIR Mental Health · doi:10.2196/16969 · Cited by 36 · cited 1×
“Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].”
ResultYin, J., Chen, X., & Xing, E. P. (2012) · PubMed · doi:10.48550/arxiv.1206.4673 · Cited by 24
Boscarino, J. A. (2015) · International Journal of Emergency Mental Health and Human Resilience · doi:10.4172/1522-4821.1000147 · Cited by 23
Community Disasters, Psychological Trauma, and Crisis Intervention.
Boscarino, J. A. (2015) · PubMed · Cited by 22
Davis, J. S. (2016) · SensePublishers eBooks · doi:10.1007/978-94-6300-702-3_3 · Cited by 14
Using Digital Data to Protect and Promote the Most Vulnerable in the Fight Against COVID-19
Chunara, R., & Cook, S. H. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00296 · Cited by 10
Koley, T. K., & Dhole, M. (2020) · Routledge eBooks · doi:10.4324/9781003095590-9 · Cited by 6
Lu, S. (2015) · PsycEXTRA Dataset · doi:10.1037/e520422015-003 · Cited by 3
Viator, F. A. (2020) · Harvard University Press eBooks · doi:10.4159/9780674245853-009 · Cited by 1
Research works printed but not linked: 5
- SadilekAKautzHSilenzioVPredicting disease transmission from geo-tagged micro-blog data2012Twenty-Sixth AAAI Conference on Artificial IntelligenceJuly 2012Toronto, ON
- CoppersmithGAHarmanCTDredzeMHMeasuring post traumatic stress disorder in Twitter2014Eighth International AAAI Conference on Weblogs and Social Media2014Ann Arbor, MI
- Saha, K, 2018, Proc Int AAAI Conf Weblogs Soc Media, 2018, 320
- EisensteinJAhmedAXingEPSparse additive generative models of text201128th International Conference on International Conference on Machine LearningJune 2011Bellevue, WA
- Aassve, A, 2020, Working Papers 661, Innocenzo Gasparini Institute for Economic Research, 1
Cited by
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Self-citationZhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581318 · Cited by 288
Social media use in disaster recovery: A systematic literature review
Ogie, R. I., James, S., Moore, A., Dilworth, T., Amirghasemi, M., & Whittaker, J. (2022) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2022.102783 · Cited by 169
John, A., Eyles, E., Webb, R. T., Okolie, C., Schmidt, L., Arensman, E., Hawton, K., O'Connor, R. C., Kapur, N., Moran, P., et al. (2021) · F1000Research · doi:10.12688/f1000research.25522.2 · Cited by 189
The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: a living systematic review
John, A., Okolie, C., Eyles, E., Webb, R. T., Schmidt, L., McGuinness, L. A., Olorisade, B. K., Arensman, E., Hawton, K., Kapur, N., et al. (2020) · F1000Research · doi:10.12688/f1000research.25522.1 · Cited by 178
Metadata
- Authors
- Saha, K., Torous, J., Caine, E. D., & De Choudhury, M.
- Affiliations
- Georgia Institute of Technology (US)
- Beth Israel Deaconess Medical Center (US)
- University of Rochester Medicine (US)
- Corresponding author
- Saha, Koustuv
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-11-24
- Volume
- 22
- Issue
- 11
- Pages
- e22600
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7690250
- OpenAlex
- W3097203256
- Semantic Scholar
- b314968db7715d78c9bc4c73972f8311d477728a
- Cited by
- 166
- References
- 95
- Citations by year
- 2026: 11 · 2025: 20 · 2024: 20 · 2023: 30 · 2022: 57 · 2021: 28
- FWCI
- 31.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (2027689)
- National Institute of Mental Health (R01MH117172)
- National Institute of General Medical Sciences (R01GM112697)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › COVID-19 and Mental Health
- Social Sciences › Communication › Communication and COVID-19 Impact
- Keywords
- COVID-19 pandemic, mental health, social media self-disclosure, Twitter, anxiety, depression, stress, suicidal ideation, emotional support, information support, temporal trends, linguistic analysis, habituation, social distancing, public health policy, United States
- MeSH terms
- COVID-19, SARS-CoV-2, Humans, Psychology, Social Media, Non-Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.