A human-machine partnered approach for identifying social media signals of elevated traumatic grief in Chicago gang territories
Open accessAbstract
There is a critical need to improve trauma-informed services in structurally marginalized communities impacted by violence and its associated traumatic grief. For community residents, particularly gang-associated youth, repeated exposure to traumatic grief causes serious adverse effects that may include negative health outcomes, delinquency, and future violent offenses. The recent proliferation of digital social media platforms, such as Twitter, provide a novel and largely underutilized resource for responding to these issues, particularly among these difficult-to-reach communities. In this paper, we explore the potential for using a human-machine partnered approach, wherein qualitative fieldwork and domain expertise is combined with a computational linguistic analysis of Twitter content among 18 gang territories/neighborhoods on Chicago's South Side. We first employ in-depth interviews and observations to identify common patterns by which residents in gang territories/neighborhoods express traumatic grief on social media. We leverage these qualitative findings, supplemented by domain expertise and computational techniques, to gather both traumatic grief- and gang-related tweets from Twitter. We next utilize supervised machine learning to construct a binary classification algorithm to eliminate irrelevant tweets that may have been gathered by our automated query and extraction techniques. Last, we confirm the validity, or ground truth, of our computational findings by enlisting additional domain expertise and further qualitative analyses of the specific traumatic events discussed in our sample of Twitter content. Using this approach, we find that social media provides useful signals for identifying moments of increased collective traumatic grief among residents in gang territories/neighborhoods. This is the first study to leverage Twitter to systematically ground the collective online articulations of traumatic grief in traumatic offline events occurring in violence-impacted communities. The results of this study will be useful for developing more effective tools-including trauma-informed intervention applications-for community organizations, violence prevention initiatives, and other public health efforts.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337 · see citing works
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,649 · see citing works
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479 · see citing works
Code of the Street: Decency, Violence, and the Moral Life of the Inner City
Halnon, K. B., & Anderson, E. (2001) · Teaching Sociology · doi:10.2307/1318956 · Cited by 3,633 · see citing works
Predicting Depression via Social Media
Choudhury, M. D., Gamon, M., Counts, S., & Horvitz, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14432 · Cited by 1,583 · see citing works
The Gang. A Study of 1,313 Gangs in Chicago
Lyon, F. E., & Thrasher, F. M. (1927) · Journal of the American Institute of Criminal Law and Criminology · doi:10.2307/1134377 · Cited by 1,226 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · cited 1× · see citing works
“This dilemma is compounded by the use of automated search methods, which often lack the capacity to process nuanced meanings and context-laden content, and run the risk of creating “automation bias” [46].”
BackgroundBig Data Surveillance: The Case of Policing
Brayne, S. (2017) · American Sociological Review · doi:10.1177/0003122417725865 · Cited by 884 · see citing works
The role of exposure to community violence and developmental problems among inner-city youth
GORMAN–SMITH, D., & TOLAN, P. (1998) · Development and Psychopathology · doi:10.1017/s0954579498001539 · Cited by 859 · see citing works
Sánchez‐Jankowski, M. (1991) · University of California Press eBooks · doi:10.1525/9780520911314 · Cited by 661 · see citing works
Surveillance and System Avoidance: Criminal Justice Contact and Institutional Attachment
Brayne, S. (2014) · American Sociological Review · doi:10.1177/0003122414530398 · Cited by 538 · see citing works
Violence and the Life Course: The Consequences of Victimization for Personal and Social Development
Macmillan, R. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.1 · Cited by 444 · see citing works
The acute effect of local homicides on children’s cognitive performance
Sharkey, P. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1000690107 · Cited by 429 · see citing works
The Corner and the Crew: The Influence of Geography and Social Networks on Gang Violence
Papachristos, A. V., Hureau, D. M., & Braga, A. A. (2013) · American Sociological Review · doi:10.1177/0003122413486800 · Cited by 427 · see citing works
Down, Out, and Under Arrest: Policing and Everyday Life in Skid Row
Stuart, F. (2016) · doi:10.7208/chicago/9780226370958.001.0001 · Cited by 388 · see citing works
Self-citationPredict and Surveil: Data, Discretion, and the Future of Policing
Brayne, S. (2020) · doi:10.1093/oso/9780190684099.001.0001 · Cited by 334 · see citing works
Tweets, Tweeps, and Signifyin’: Communication and Cultural Performance on “Black Twitter”
Florini, S. (2014) · Television & New Media · doi:10.1177/1527476413480247 · Cited by 271 · see citing works
Zinzow, H. M., Ruggiero, K. J., Resnick, H. S., Hanson, R. F., Smith, D. W., Saunders, B. E., & Kilpatrick, D. G. (2009) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2008.02004.x · Cited by 264 · see citing works
The Ethical Algorithm: The Science of Socially Aware Algorithm Design
Kearns, M. J., & Roth, A. (2021) · Perspectives on Science and Christian Faith · doi:10.56315/pscf3-21kearns · Cited by 261 · see citing works
Social Networks and the Risk of Gunshot Injury
Papachristos, A. V., Braga, A. A., & Hureau, D. M. (2012) · Journal of Urban Health · doi:10.1007/s11524-012-9703-9 · Cited by 231 · see citing works
Bell, C. C., & Jenkins, E. J. (1991) · Journal of Health Care for the Poor and Underserved · doi:10.1353/hpu.2010.0089 · Cited by 224 · see citing works
A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media
Chancellor, S., Birnbaum, M. L., Caine, E. D., Silenzio, V. M. B., & De Choudhury, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287587 · Cited by 214 · see citing works
Black Picket Fences, Second Edition: Privilege and Peril among the Black Middle Class
Pattillo, M., & Lareau, A. (2013) · doi:10.7208/chicago/9780226021225.001.0001 · Cited by 201 · see citing works
Knowing the Tweeters: Deriving Sociologically Relevant Demographics from Twitter
Sloan, L., Morgan, J., Housley, W., Williams, M. L., Edwards, A., Burnap, P., & RANA, O. F. (2013) · Sociological Research Online · doi:10.5153/sro.3001 · Cited by 197 · see citing works
Stress, Glucocorticoids, and Memory: A Meta-analytic Review
Sauro, M. D., Jorgensen, R. S., & Pedlow, C. T. (2003) · Stress · doi:10.1080/10253890310001616482 · Cited by 184 · see citing works
BEREAVEMENT BY TRAUMATIC MEANS: THE COMPLEX SYNERGY OF TRAUMA AND GRIEF
Neria, Y., & Litz, B. T. (2004) · Journal of Loss and Trauma · doi:10.1080/15325020490255322 · Cited by 172 · see citing works
Collateral Consequences of Violence in Disadvantaged Neighborhoods
Harding, D. J. (2009) · Social forces; a scientific medium of social study and interpretation · doi:10.1353/sof.0.0281 · Cited by 161 · see citing works
Evaluation of CeaseFire-Chicago
Skogan, W. G., Hartnett, S. M., & Bump, N. (2012) · Cited by 150 · see citing works
THE ECOLOGY OF GANG TERRITORIAL BOUNDARIES*
Brantingham, P. J., Tita, G., Short, M. B., & Reid, S. E. (2012) · Criminology · doi:10.1111/j.1745-9125.2012.00281.x · Cited by 147 · see citing works
“Code of the Tweet: Urban Gang Violence in the Social Media Age.”
Stuart, F. (2020) · Social Problems · doi:10.1093/socpro/spz010 · Cited by 128 · see citing works
Self-citationBallad of the bullet: gangs, drill music, and the power of online infamy
James, M. (2020) · Ethnic and Racial Studies · doi:10.1080/01419870.2020.1813320 · Cited by 125 · see citing works
Haskins, A. R., & Jacobsen, W. C. (2017) · American Sociological Review · doi:10.1177/0003122417709294 · Cited by 124 · see citing works
Lane, J. (2018) · doi:10.1093/oso/9780199381265.001.0001 · Cited by 124 · see citing works
On the Validity and Reliability of Gang Homicide: A Comparison of Disparate Sources
Decker, S. H., & Pyrooz, D. C. (2010) · Homicide Studies · doi:10.1177/1088767910385400 · Cited by 111 · see citing works
#Proana: Pro-Eating Disorder Socialization on Twitter
Arseniev‐Koehler, A., Lee, H. E., McCormick, T. H., & Moreno, M. A. (2016) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2016.02.012 · Cited by 109 · see citing works
Saving Children, Controlling Families: Punishment, Redistribution, and Child Protection
Edwards, F. (2016) · American Sociological Review · doi:10.1177/0003122416638652 · Cited by 106 · see citing works
Patton, D. U., Lane, J., Leonard, P., Macbeth, J., & Lee, J. R. S. (2016) · New Media & Society · doi:10.1177/1461444815625949 · Cited by 94 · see citing works
Traumatic Loss: Conceptual and Empirical Links Between Trauma and Bereavement
Green, B. L. (2000) · Journal of Personal and Interpersonal Loss · doi:10.1080/10811440008407845 · Cited by 82 · see citing works
Daipha, P. (2015) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226298719.001.0001 · Cited by 72 · see citing works
Patton, D. U., Brunton, D.-W., Dixon, A. L., Miller, R. J., Leonard, P., & Hackman, R. (2017) · Social Media + Society · doi:10.1177/2056305117733344 · Cited by 66 · see citing works
Getting to Know You: Welfare Fraud Investigation and the Appropriation of Social Ties
Headworth, S. (2019) · American Sociological Review · doi:10.1177/0003122418818198 · Cited by 55 · see citing works
Karamshuk, D., Shaw, F., Brownlie, J., & Sastry, N. (2017) · Online Social Networks and Media · doi:10.1016/j.osnem.2017.01.002 · Cited by 52 · see citing works
Cure Violence: Treating Violence As a Contagious Disease
Slutkin, G., Ransford, C., & Decker, R. B. (2015) · Springer eBooks · doi:10.1007/978-3-319-15868-6_5 · Cited by 46 · see citing works
Patton, D. U., Macbeth, J., Schoenebeck, S. Y., Shear, K. M., & McKeown, K. R. (2018) · Biomedical Informatics Insights · doi:10.1177/1178222618763155 · Cited by 43 · see citing works
Allen, K., Hollingworth, S., Mansaray, A. A., & Taylor, Y. (2013) · Sociological Research Online · doi:10.5153/sro.3149 · Cited by 25 · see citing works
Patton, D. U., Sanchez, N., Fitch, D., Macbeth, J., & Leonard, P. (2015) · Social Science Computer Review · doi:10.1177/0894439315613319 · Cited by 24 · see citing works
Expressions of loss predict aggressive comments on Twitter among gang-involved youth in Chicago
Patton, D. U., Rambow, O., Auerbach, J., Li, K., & Frey, W. R. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0020-x · Cited by 18 · see citing works
Fine, G. A. (2005) · Journal of Contemporary Ethnography · doi:10.1177/0891241605282241 · Cited by 7 · see citing works
Patton, D. U., McKeown, K., Rambow, O., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08779 · Cited by 6 · see citing works
The metonymy of #BlackTwitter: The effect of urban identification on hashtag activism
James, J. P. (2019) · Journal of cultural marketing strategy. · doi:10.69554/nnzc9770 · Cited by 2 · see citing works
Chicago Gang Social Media Data Set
Patton, D. U., & Macbeth, J. (2016) · doi:10.7916/d84f1r07 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [9]Abreu A, Acker A. Context and collection: A research agenda for small data. In: iConference 2013 proceedings. Fort Worth, Texas; 2013. pp. 549–554.
- [11]Sered D. Until we reckon: Violence, mass incarceration, and a road to repair. New York, NY: The New Press; 2019.
- [12]Rich JA. Wrong place, wrong time: Trauma and violence in the lives of young black men. Baltimore, MD: John Hopkins University Press; 2009.
Cited by
Sorted by citations
Gang Research in the Twenty-First Century
Moore, C. L., & Stuart, F. D. (2021) · Annual Review of Criminology · doi:10.1146/annurev-criminol-030920-094656 · Cited by 44 · see citing works
Self-citationElSherief, M., Saha, K., Gupta, P., Mishra, S., Seybolt, J., Xie, J., O’Toole, M., Burd-Sharps, S., & De Choudhury, M. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00993-6 · Cited by 43 · see citing works
“Likewise, other recent research has used qualitative data, such as that from field work and domain experts, to enhance the validity or ground truth of computational linguistic analyses (Stuart et al., 2020).”
Fernández-Planells, A., Orduña‐Malea, E., & Feixa, C. (2021) · New Media & Society · doi:10.1177/1461444821994490 · Cited by 31 · see citing works
Lane, J., & Stuart, F. D. (2022) · Qualitative Sociology · doi:10.1007/s11133-022-09510-w · Cited by 18 · see citing works
Self-citationHow Americans encounter guns: Mixed methods content analysis of YouTube and internet search data
Ranney, M. L., Conrey, F. R., Perkinson, L., Friedhoff, S., Smith, R., & Wardle, C. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107258 · Cited by 11 · see citing works
Patton, D. U., Aguilar, N. J., Landau, A. Y., Thomas, C., Kagan, R., Ren, T., Stoneberg, E., Wang, T., Halmos, D., Saha, A., et al. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107263 · Cited by 9 · see citing works
Lane, J., & Marler, W. (2020) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197510636.013.41 · Cited by 4 · see citing works
DIGITAL VIOLENCE AND WOMEN: SYSTEMATIZATION OF RESEARCHS AND SUGGESTIONS FOR FUTURE RESEARCH
DUMAN, M. Ç. (2023) · Anadolu Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi · doi:10.53443/anadoluibfd.1187094 · Cited by 8 · see citing works
Re-Examining the Literature on Social Media and Gangs
Elsaesser, C. M., & Patton, D. U. (2024) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197618158.013.20
Tucker, R., Rim, N., Chao, A., Gaillard, E. P., & Berman, M. G. (2025) · Journal of Quantitative Criminology · doi:10.1007/s10940-025-09615-9
Fowler, S. L., Stylianou, A. C., Zhang, D., Lowry, P. B., Mousavi, R., & Reid, S. E. (2026) · Information & Management · doi:10.1016/j.im.2026.104345
Warner, T. D., Braman, A., Mullins, J., & Warner, G. (2026) · Mortality · doi:10.1080/13576275.2026.2662580
AI in Advocacy and Social Justice
Alba, C., & McCoy, H. (2026) · Springer eBooks · doi:10.1007/978-3-032-18443-6_16
- Authors
- Stuart, F., Riley, A., & Pourreza, H.
- Affiliations
- Stanford University (US)
- University of California, San Francisco (US)
- University of Chicago (US)
- Corresponding author
- Stuart, Forrest
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-07-30
- Volume
- 15
- Issue
- 7
- Pages
- e0236625
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7392535
- OpenAlex
- W3046853450
- Cited by
- 13
- References
- 51
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 1 · 2023: 1 · 2022: 3 · 2021: 3 · 2020: 1
- FWCI
- 2.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- eScholarship (California Digital Library) (submittedVersion)
- University of Chicago (cc-by)
- Funders
- University of Chicago Women's Board
- University of Chicago Urban Health Initiative
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Clinical Psychology › Grief, Bereavement, and Mental Health
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- traumatic grief, social media, Twitter, human-machine collaboration, computational linguistics, trauma-informed services, violence prevention, Chicago South Side, collective grief, tweet classification
- MeSH terms
- Algorithms, Chicago, Grief, Humans, Residence Characteristics, Violence, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review