The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index; 5 of the 38 sources its claims cite were checked for support.
Impacts of school shooter drills on the psychological well-being of American K-12 school communities: a social media study
Open accessAbstract
The toll from gun violence in American K-12 schools has escalated over the past 20 years. School administrators face pressure to prepare for possible active shootings, and often do so through drills, which can range from general lockdowns to simulations, involving masked “shooters” and simulated gunfire, and many variations in between. However, the broad and lasting impact of these drills on the well-being of school communities is poorly understood. To that end, this article applies machine learning and interrupted time series analysis to 54 million social media posts, both pre- and post-drills in 114 schools spanning 33 states. Drill dates and locations were identified via a survey, then posts were captured by geo-location, school social media following, and/or school social media group membership. Results indicate that anxiety, stress, and depression increased by 39–42% following the drills, but this was accompanied by increases in civic engagement (10–106%). This research, paired with the lack of strong evidence that drills save lives, suggests that proactive school safety strategies may be both more effective, and less detrimental to mental health, than drills.
Overview
CoChat Summary
TL;DR
School shooter drills were associated with sustained increases in anxiety/stress and depression markers in social media posts from K-12 school communities, alongside increases in community/civic engagement. The authors argue that current unregulated drills may harm mental health and that proactive safety strategies may be preferable.
Key findings
- Anxiety/stress and depression expressions rose after drills: mean proportion of stress/anxiety posts was 0.281 before vs. 0.399 after (LC = 42.1% increase; t = 19.1, p < 10⁻¹⁵), and depression was 0.125 vs. 0.173 (LC = 38.7%; t = 10.13, p < 10⁻¹⁵), with effects sustained at least 90 days.
- Community/civic outcomes also increased: the abstract reports civic engagement increases of 10–106%; results show LIWC increases in perception (10.78%), article (16.3%), first-person plural pronouns (47.79%), second-person pronouns (10.21%), friends (33.7%), and work (106.18%).
- Robustness checks supported attribution to drills: excluding mass-shooting overlaps still yielded LC increases of 43.12% for stress/anxiety and 39.58% for depression (temporal exclusion) and 30% and 29.6% (spatial exclusion); ARIMA counterfactual predicted 26.3% and 50% lower stress/anxiety and depression than actual; actual control time series changed only −0.375–2.5%.
- Subgroup analyses found high school communities and teachers showed the largest changes: ↑24.3–55.1% for anxiety/stress and depression and ↑5.9–27.2% for LIWC categories (p < 0.001).
Methods
- Observational natural-experiment design using social media, with interrupted time series (ITS) analysis to estimate immediate change (IC) and longer-term change (LC) around drill dates, plus four robustness checks: temporal/spatial exclusion of mass shootings, ARIMA counterfactual forecasting, synthetic-control permutation tests, and an actual Twitter control.
- Data: a survey fielded November 2019–February 2020 from 153 volunteers identified drill dates/locations; final analyses covered 114 schools in 33 states and 54 million public Twitter/Reddit posts (27.8M treatment + ~27M control) spanning 90 days before and after each drill, January 2018–March 2020; focus groups included 34 stakeholders (21 parents, 11 teachers, 2 students) in 6 groups.
- Analysis: SVM classifiers for stress/anxiety and depression, LIWC psycholinguistic categories, SAGE n-gram saliency, Welch t-tests with Benjamini-Krieger-Yekutieli FDR correction, and inductive qualitative coding of focus-group transcripts.
Limitations
- Stated: classifiers are not clinical diagnostic tools; true causality cannot be inferred; the study covers only a 3-month post-drill window; social media data carry self-selection bias; Twitter excludes users under 13; the survey did not collect drill type or implementation; focus groups cannot provide the same quantitative validity as large-scale data.
- Evident from the text: drill dates came from a self-reported survey of volunteers in a grassroots gun violence prevention organization, so the sample may not represent all schools or communities; social media data miss non-users and younger students; and the study does not directly measure individual drill exposure or participation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 27 Not checked · 84 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The fourth method involved comparing our temporal trends (treatment), speci fi c to the schools and their respective drill dates, with a suitably chosen control time series in the same timeframe (De Choudhury and Kiciman, 2017).”
What the sources say
- Disputedcites De Choudhury and Kiciman, 2017 sourceExperimental — accuracy not yet measured
The Data and Method passages describe Reddit users in mental-health communities, not school-specific Twitter data or drill dates. The Method defines treatment as receiving a comment token and control as not receiving it; it does not compare temporal trends against a non-school time series. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“We quanti fi ed symptomatic expressions of stress, anxiety, and depression using machine-learning classi fi ers that were built and validated in prior work (Saha et al., 2019b).”
What the sources say
- Supportedcites Saha et al., 2019b source
In Methods > Defining and Measuring Symptomatic Outcomes, the paper says it built classifiers for depression, anxiety, and stress. The same section reports cross-validation and test accuracy, plus a manual check of classifier labels against Twitter posts. This supports the citing paper’s description of classifiers built and validated in prior work. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This tool is known to work well with short text and social media data, as revealed in a large body of social computing and crisis informatics literature (Lin and Margolin, 2014; Mark et al., 2012; Saha and De Choudhury, 2017).”
What the sources say
- Partly supportedcites Lin and Margolin, 2014 sourceExperimental — accuracy not yet measured
Section 2.3 (page unavailable) says LIWC Dictionary is an applicable approach for analyzing tweets, which supports its relevance to social media text. The paper uses SentiSense instead and does not establish that LIWC works well with short text or social media data generally. (LLM, conf=0.83)
- Not checked — no full text of the sourcecites Mark et al., 2012 source
- Not checked — no full text of the sourcecites Saha and De Choudhury, 2017 source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We also acknowledge that our quantitative analyses may suffer from self-selection bias, as it is restricted to only those who are on social media and choose to post on it (Olteanu et al., 2019).”
What the sources say
- Supportedcites Olteanu et al., 2019 source
Section 3.2 (Population Biases; page not supplied) says people do not randomly self-select into social media use. Section 3.3 (Behavioral Biases; page not supplied) says self-selection bias can arise from what users choose to share and notes that a dataset may exclude people who read without posting. Together, these support the cited limitation. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
None of the sources cited for this claim have been checked yet.
In the paper
“Notwithstanding these negative impacts, the fi nding around increased collective action and advocacy resonates with the trend whereby school shooting incidents and the fear surrounding them propels many to join the broader movement of gun violence prevention (Everytown, 2019).”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
39 more claims
Create a free account to see all 117 claims and every source verdict.
- It may state two separate claims as one. 51
- Probably not a substantial claim. 31
- The quote only partly states this claim. 3
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Results · p. 9
“p = 0 for stress/anxiety, depression, article, first-person plural, second person , and work”
§ Results · p. 9
“p = 0.035 for second person )”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Data and methods · p. 5
“This led to the identi fi cation of eight mass shooting events in nine states”
§ Data and methods · p. 5
“These states were Kentucky, Florida, Texas, North Carolina, Colorado, Georgia, California, and Louisiana.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Discussion · p. 11
“Therefore, drills occurring at least 90 days apart -not the whole 6 months -did not overlap with the pre- or post-periods and thus were unlikely to impact results.”
§ Data and methods · p. 3
“spanning a 90-day period before (~3 months) and a 90-day period after (~3 months) after each drill identi fi ed above.”
Citations
Cites
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“These are essentially n-gram based binary support vector machine (SVM) models trained using transfer learning methodologies (Pan and Yang, 2009)—transfer learning is a method where a model developed for a task is reused as the starting point for a model on a second task.”
MethodBirds of a Feather: Homophily in Social Networks
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.415 · Cited by 18,567 · cited 1×
“The first approach uses the homophily principle that structures network ties such as those relating to work, support, and other relationships (McPherson et al., 2001).”
BackgroundStress, social support, and the buffering hypothesis
Cohen, S., & Wills, T. A. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.98.2.310 · Cited by 18,093 · cited 1×
“Open coding is common in the analysis of qualitative research and is an established approach in Grounded Theory (Charmaz, 2014).”
MethodThe Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods
Tausczik, Y. R., & Pennebaker, J. W. (2010) · Journal of Language and Social Psychology · doi:10.1177/0261927x09351676 · Cited by 5,774 · cited 12×
“Similarly, they manifest in many aspects of our lives, including our general thought processes (Palen and Anderson, 2016; Tausczik and Pennebaker, 2010).”
BackgroundMethodInfluentialboyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Interrupted time series regression for the evaluation of public health interventions: a tutorial
Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw098 · Cited by 3,049
Psychological Aspects of Natural Language Use: Our Words, Our Selves
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145041 · Cited by 2,624 · cited 2×
“Language can reflect these affective, cognitive, perceptual, and other psychological processes of individuals, including changes over time around specific events (school shooter drills in our case) (Pennebaker et al., 2003).”
BackgroundA Companion to Qualitative Research
Schofield, P. (2005) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2005.03486_1.x · Cited by 1,362
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Research works printed but not linked: 7
- Beaton RDMurphy SA,(1995) Working with people in crisis: Research implications. In: Compassion fatigue: coping with secondary traumatic stress disorder in those who treat the traumatized. vol. 1. Brunner/Routledge, pp. 1–51
- Eisenstein J, Ahmed A, Xing EP (2011) Sparse additive generative models of text. In Proceedings of the 28th International Conference on Machine Learning, ICML, pp. 1041–1048
- Farrell A (2005) Ethical research with children. McGraw-Hill Education (UK)
- Gubiotti M (2015) Opposing viewpoints: preparing students, teachers, and the community for school shootings: Saving lives with active shooter simulations. Child Legal Right J 35(3):254
- Hurford DP, Lindskog R, Cole A, Jackson R, Thomasson S, Wade A (2010) The role of school climate in school violence: a validity study of a web-based school violence survey. J Educ Res Policy Stud 10(1):51–77
- Rygg L (2015) School shooting simulations: At what point does preparation become more harmful than helpful. Child Legal Right J 35:215
- Terrades V, Khan SK (2018) Will it ever end: Preventing mass shootings in florida & the us. Suffolk Univ Law Rev 51:505
Cited by
How Mass Public Shooters Use Social Media: Exploring Themes and Future Directions
Peterson, J., Densley, J. A., Spaulding, J. S., & Higgins, S. (2023) · Social Media + Society · doi:10.1177/20563051231155101 · Cited by 31
“Prior research has examined how the general public interacts with mass shootings on social media, typically through the lens of collective trauma (ElSherief et al., 2021; Thompson et al., 2019), but not necessarily how mass shooters themselves interact with social media.”
BackgroundEpidemiology of Mass Shootings in the United States
Peterson, J. K., Densley, J. A., Hauf, M., & Moldenhauer, J. (2024) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-081122-010256 · Cited by 23
The Pursuit of Peer Support for Opioid Use Recovery on Reddit
Balsamo, D., Bajardi, P., Morales, G. D. F., Monti, C., & Schifanella, R. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22122 · Cited by 31
Rajan, S., Reeping, P. M., Ladhani, Z., Vasudevan, L. M., & Branas, C. C. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107280 · Cited by 23
Quantifying Pediatric Gun Violence by Location, Time of Day, and Day of Week
Ordoobadi, A. J., Wickard, A., Heindel, P. V., Raykar, N. P., Masiakos, P. T., & Anderson, G. A. (2023) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2023.11.002 · Cited by 11
Metadata
- Authors
- ElSherief, M., Saha, K., Gupta, P., Mishra, S., Seybolt, J., Xie, J., O’Toole, M., Burd-Sharps, S., & De Choudhury, M.
- Affiliations
- University of California San Diego (US)
- Microsoft Research Montréal (Canada) (CA)
- Georgia Institute of Technology (US)
- Venue
- Humanities and Social Sciences Communications
- Published
- 2021-12-08
- Volume
- 8
- Issue
- 1
- ISSN
- 2055-1045 (ISSN-L)
- 2662-9992
- OpenAlex
- W4200177403
- Cited by
- 43
- References
- 60
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 12 · 2023: 10 · 2022: 5
- FWCI
- 3.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Humanities and Social Sciences Communications (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Georgia Institute of Technology
- Topics
- Social Sciences › Health › Gun Ownership and Violence Research
- Social Sciences › Psychology › Clinical Psychology › Posttraumatic Stress Disorder Research
- Social Sciences › Safety Research › Education, Safety, and Science Studies
- Keywords
- school safety, psychological well-being, anxiety, stress, depression, citizen participation, school community, social media posts, K-12 schools, school lockdown
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Impacts of school shooter drills on the psychological well-being of American K-12 school communities: a social media study. https://app.cochat.ai/papers/10.1057/s41599-021-00993-6
- The paper
- ElSherief, M., Saha, K., Gupta, P., Mishra, S., Seybolt, J., Xie, J., O’Toole, M., Burd-Sharps, S., & De Choudhury, M. (2021). Impacts of school shooter drills on the psychological well-being of American K-12 school communities: a social media study. Humanities and Social Sciences Communications. https://doi.org/10.1057/s41599-021-00993-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.