Automated stratification of trauma injury severity across multiple body regions using multi-modal, multi-class machine learning models
Open accessPublished version of a preprint
Abstract
Objective: The timely stratification of trauma injury severity can enhance the quality of trauma care but it requires intense manual annotation from certified trauma coders. The objective of this study is to develop machine learning models for the stratification of trauma injury severity across various body regions using clinical text and structured electronic health records (EHRs) data. Materials and Methods: Our study utilized clinical documents and structured EHR variables linked with the trauma registry data to create 2 machine learning models with different approaches to representing text. The first one fuses concept unique identifiers (CUIs) extracted from free text with structured EHR variables, while the second one integrates free text with structured EHR variables. Temporal validation was undertaken to ensure the models’ temporal generalizability. Additionally, analyses to assess the variable importance were conducted. Results: Both models demonstrated impressive performance in categorizing leg injuries, achieving high accuracy with macro-F1 scores of over 0.8. Additionally, they showed considerable accuracy, with macro-F1 scores exceeding or near 0.7, in assessing injuries in the areas of the chest and head. We showed in our variable importance analysis that the most important features in the model have strong face validity in determining clinically relevant trauma injuries. Discussion: The CUI-based model achieves comparable performance, if not higher, compared to the free-text-based model, with reduced complexity. Furthermore, integrating structured EHR data improves performance, particularly when the text modalities are insufficiently indicative. Conclusions: Our multi-modal, multiclass models can provide accurate stratification of trauma injury severity and clinically relevant interpretations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Machine learning models that integrate task-relevant multimodal EHR yield superior performance compared to those relying on a single modality.
“Machine learning models that integrate task-relevant multimodal EHR have been shown to yield superior performance compared to those relying on a single modality2–6.”
From MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling (2025) · cites this paper as [5]
Section 3.3 (page not provided) reports that adding structured EHR data significantly improved performance for arm and extremity injuries. It did not produce a significant improvement in the other three body regions, so this paper does not support the claim of generally superior multimodal performance. (LLM, conf=0.89)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Baker, S. P., O’Neill, B., Haddon, W. F., & Long, W. B. (1974) · PubMed · Cited by 7,946 · see citing works
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906 · see citing works
Baker, S. P., O’Neill, B., Haddon, W. F., & Long, W. B. (1974) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/00005373-197403000-00001 · Cited by 7,326 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · see citing works
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,359 · see citing works
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
Self-citationSavova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · see citing works
Alsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., & McDermott, M. (2019) · Proceedings of the 2nd Clinical Natural Language Processing Workshop · doi:10.18653/v1/w19-1909 · Cited by 1,736 · see citing works
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · see citing works
(2009) · PEDIATRICS · doi:10.1542/peds.2009-0257 · Cited by 800 · see citing works
Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers
Binder, A., Montavon, G., Lapuschkin, S., Müller, K., & Samek, W. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-44781-0_8 · Cited by 455 · see citing works
National Inventory of Hospital Trauma Centers
MacKenzie, E. J., Hoyt, D. B., Sacra, J. C., Jurkovich, G. J., Carlini, A. R., Teitelbaum, S. D., & Teter, H. (2003) · JAMA · doi:10.1001/jama.289.12.1515 · Cited by 312 · see citing works
The value of trauma registries
Moore, L., & Clark, D. E. (2008) · Injury · doi:10.1016/j.injury.2008.02.023 · Cited by 294 · see citing works
Defining major trauma using the 2008 Abbreviated Injury Scale
Palmer, C. S., Gabbe, B. J., & Cameron, P. A. (2016) · Injury · doi:10.1016/j.injury.2015.07.003 · Cited by 263 · see citing works
Trauma severity scoring to predict mortality
Champion, H. R., Sacco, W. J., & Hunt, T. K. (1983) · World Journal of Surgery · doi:10.1007/bf01655906 · Cited by 217 · see citing works
External validation of AI models in health should be replaced with recurring local validation
Youssef, A., Pencina, M., Thakur, A., Zhu, T., Clifton, D., & Shah, N. H. (2023) · Nature Medicine · doi:10.1038/s41591-023-02540-z · Cited by 170 · see citing works
The Trauma Registry as a Statewide Quality Improvement Tool
Cameron, P. A., Gabbe, B. J., McNeil, J. J., Finch, C. F., Smith, K. L., Cooper, D. J., Judson, R., & Kossmann, T. (2005) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000198350.15936.a1 · Cited by 165 · see citing works
Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z., & Guyon, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.10201 · Cited by 158 · see citing works
Friedman, Z., Kugel, C., Hiss, J., Marganitt, B., Stein, M., & Shapira, S. C. (1996) · American Journal of Forensic Medicine & Pathology · doi:10.1097/00000433-199609000-00011 · Cited by 127 · see citing works
He, Y., Zhu, Z., Zhang, Y., Chen, Q., & Caverlee, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.372 · Cited by 111 · see citing works
Ye, J., Yao, L., Shen, J., Janarthanam, R., & Luo, Y. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01318-4 · Cited by 105 · see citing works
Using AI to measure Parkinson’s disease severity at home
Islam, M. S., Rahman, W., Abdelkader, A., Lee, S., Yang, P. T., Purks, J. L., Adams, J. L., Schneider, R. B., Dorsey, E. R., & Hoque, M. E. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00905-9 · Cited by 99 · see citing works
Epidemiology of in-hospital trauma deaths
Lefering, R. L., Paffrath, T., Bouamra, O., Coats, T., Woodford, M., Jenks, T., Wafaisade, A., Nienaber, U., & Lecky, F. E. (2011) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-011-0168-4 · Cited by 89 · see citing works
Use of unstructured text in prognostic clinical prediction models: a systematic review
Seinen, T. M., Fridgeirsson, E. A., Ioannou, S., Jeannetot, D. Y. B., John, L. H., Kors, J. A., Markus, A. F., Pera, V., Rekkas, A., Williams, R. D., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac058 · Cited by 80 · see citing works
WHO releases Guidelines for trauma quality improvement programmes
Mock, C. (2009) · Injury Prevention · doi:10.1136/ip.2009.024315 · Cited by 51 · see citing works
From Survival to Survivorship — Framing Traumatic Injury as a Chronic Condition
Herrera‐Escobar, J. P., & Schneider, J. C. (2022) · New England Journal of Medicine · doi:10.1056/nejmp2205162 · Cited by 50 · see citing works
Afshar, M., Adelaine, S., Resnik, F., Mundt, M. P., Long, J., Leaf, M., Ampian, T., Wills, G. J., Schnapp, B., Chao, M., et al. (2023) · JMIR Medical Informatics · doi:10.2196/44977 · Cited by 44 · see citing works
Self-citationMachine Learning in the Prediction of Trauma Outcomes: A Systematic Review
Zhang, T., Nikouline, A., Lightfoot, D., & Nolan, B. (2022) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2022.05.011 · Cited by 42 · see citing works
Predicting Clinical Outcomes Across Changing Electronic Health Record Systems
Gong, J. J., Naumann, T., Szolovits, P., & Guttag, J. V. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098064 · Cited by 34 · see citing works
Afshar, M., Dligach, D., Sharma, B., Cai, X., Boyda, J., Birch, S., Valdez, D., Zelisko, S., Joyce, C., Modave, F. P., et al. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz068 · Cited by 34 · see citing works
Self-citationA criteria-directed protocol for in-hospital triage of trauma patients
Granström, A., Strömmer, L., Schandl, A. R., & Östlund, A. (2016) · European Journal of Emergency Medicine · doi:10.1097/mej.0000000000000397 · Cited by 33 · see citing works
Kulshrestha, S., Dligach, D., Joyce, C., Baker, M. S., Gonzalez, R., O’Rourke, A. P., Glazer, J. M., Stey, A. M., Kruser, J. M., Churpek, M. M., et al. (2020) · Injury · doi:10.1016/j.injury.2020.10.094 · Cited by 33 · see citing works
Self-citationComparison and interpretability of machine learning models to predict severity of chest injury
Kulshrestha, S., Dligach, D., Joyce, C., Gonzalez, R., O’Rourke, A. P., Glazer, J. M., Stey, A. M., Kruser, J. M., Churpek, M. M., & Afshar, M. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab015 · Cited by 12 · see citing works
Self-citationLeveraging Clinical BERT in Multimodal Mortality Prediction Models for COVID-19
Pawar, Y., Henriksson, A., Hedberg, P., & Nauclér, P. (2022) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms55023.2022.00042 · Cited by 12 · see citing works
Measuring Pain in Sickle Cell Disease using Clinical Text
Alambo, A., Andrew, R., Gollarahalli, S., Vaughn, J., Banerjee, T., Thirunarayan, K., Abrams, D. M., & Shah, N. (2020) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc44109.2020.9175599 · Cited by 11 · see citing works
Fanconi, C., van Buchem, M. M., & Hernandez‐Boussard, T. (2023) · PubMed Central · Cited by 9 · see citing works
Fanconi, C., van Buchem, M. M., & Hernandez‐Boussard, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.13860 · Cited by 6 · see citing works
Evaluation of crowdsourced mortality prediction models as a framework for assessing AI in medicinePreprint
Bergquist, T. R., Schaffter, T., Yan, Y., Yu, T. V., Prosser, J., Gao, J., Chen, G., Charzewski, Ł., Nawalany, Z., Brugere, I., et al. (2021) · medRxiv · doi:10.1101/2021.01.18.21250072 · Cited by 5 · see citing works
Self-citationTowards Generalizable Methods for Automating Risk Score Calculation
Liang, J. J., Lehman, E., Iyengar, A. S., Mahajan, D., Raghavan, P., Chang, C. Y., & Szolovits, P. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.42 · Cited by 5 · see citing works
A hybrid system to understand the relations between assessments and plans in progress notes
Gao, J., He, S., Hu, J., & Chen, G. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104363 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 5
- Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805. 2018.
- Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks. In: Interna- tional conference on machine learning. PMLR; 2017. p. 3319-28.
- Shrikumar A, Greenside P, Kundaje A. Learning important features through propagating activation differences. In: International conference on machine learning. PMLR; 2017. p. 3145-53.
- Aronson AR. Metamap: Mapping text to the umls metathesaurus. Bethesda, MD: NLM, NIH, DHHS. 2006;1:26.
- Asensio JA, Trunkey DD. Current Therapy of Trauma and Surgical Critical Care E-Book. Elsevier Health Sciences; 2008.
Cited by
Sorted by citations
Teles, A. S., de Moura, I. R., Silva, F., Roberts, A., & Stahl, D. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102981 · Cited by 22 · see citing works
Lee, A., Chan, M. K., Narayanan, D., Staudenmayer, K., Nassar, A. K., Forrester, J. D., Knowlton, L. M., & Hameed, S. M. (2025) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000004805 · Cited by 5 · see citing works
MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling
Gao, J., Rahman, M., Caskey, J., Oguss, M., O’Rourke, A., Brown, R., Stey, A., Mayampurath, A., Churpek, M. M., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02219-4 · Cited by 2 · see citing works
Self-citationPredicting Trauma Severity from Imbalanced Data Using Ensemble Regression and Generative Models
Rahman, S., Rahman, M. M., Ridelman, E., Frei, A., Adams, R. M., Rastgaar, M., Shanti, C. M., & Faezipour, M. (2025) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi67747.2025.11269452
Lin, Y.-J., Fang, K.-Y., Liao, C., & Cheng, C. (2026) · JMIR Medical Informatics · doi:10.2196/93922
- Authors
- Gao, J., Chen, G., O’Rourke, A. P., Caskey, J., Carey, K. A., Oguss, M., Stey, A., Dligach, D., Miller, T., Mayampurath, A., Churpek, M. M., & Afshar, M.
- Affiliations
- University of Wisconsin–Madison (US)
- Northwestern University (US)
- Loyola University Chicago (US)
- Boston Children's Hospital (US)
- Corresponding author
- Afshar, Majid
- ORCID
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-04-08
- Volume
- 31
- Issue
- 6
- Pages
- 1291-1302
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC11105131
- OpenAlex
- W4394563567
- Cited by
- 5
- References
- 45
- Citations by year
- 2026: 1 · 2025: 4
- FWCI
- 0.74
- Type
- journal article
- Language
- en
- Open access
- green
- dx.doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- medRxiv (acceptedVersion, cc-by-nd)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute on Drug Abuse (R01DA051464)
- National Institute of General Medical Sciences (R01HL157262)
- National Library of Medicine (K23HL157832, R01LM010090, R01LM012793)
- University of Wisconsin School of Medicine and Public Health
- Wisconsin Partnership Program
- Research Design Support: the Protocol Development, Informatics, and Biostatistics Module
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- clinical text, electronic health records, multimodal learning, leg injuries, thoracic trauma, traumatic brain injury, trauma registry, temporal validation, feature importance
- MeSH terms
- Machine Learning, Humans, Natural Language Processing, Registries, Wounds and Injuries, Trauma Severity Indices, Injury Severity Score, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
