Machine learning classification of new firearm injury encounters in the St Louis region: 2010-2020
Open accessAbstract
Objectives: To improve firearm injury encounter classification (new vs follow-up) using machine learning (ML) and compare our ML model to other common approaches. Materials and Methods: This retrospective study used data from the St Louis region-wide hospital-based violence intervention program data repository (2010-2020). We randomly selected 500 patients with a firearm injury diagnosis for inclusion, with 808 total firearm injury encounters split (70/30) for training and testing. We trained a least absolute shrinkage and selection operator (LASSO) regression model with the following predictors: admission type, time between firearm injury visits, number of prior firearm injury emergency department (ED) visits, encounter type (ED or other), and diagnostic codes. Our gold standard for new firearm injury encounter classification was manual chart review. We then used our test data to compare the performance of our ML model to other commonly used approaches (proxy measures of ED visits and time between firearm injury encounters, and diagnostic code encounter type designation [initial vs subsequent or sequela]). Performance metrics included area under the curve (AUC), sensitivity, and specificity with 95% confidence intervals (CIs). Results: The ML model had excellent discrimination (0.92, 0.88-0.96) with high sensitivity (0.95, 0.90-0.98) and specificity (0.89, 0.81-0.95). AUC was significantly higher than time-based outcomes, sensitivity was slightly (but not significantly) lower than other approaches, and specificity was higher than all other methods. Discussion: ML successfully delineated new firearm injury encounters, outperforming other approaches in ruling out encounters for follow-up. Conclusion: ML can be used to identify new firearm injury encounters and may be particularly useful in studies assessing re-injuries.
AI snapshot
Written from the paper’s full text.
TL;DR
A LASSO machine learning model using encounter and diagnostic-code features classified new versus follow-up firearm injury encounters with high discrimination and specificity, outperforming time-based rules and matching or exceeding ED-visit and ICD-10 encounter-type proxies on several metrics. The authors conclude ML can improve identification of new firearm injury encounters, especially for ruling out follow-up care.
Key findings
- In the test set (N=241), the ML model had AUC 0.92 (0.88–0.96), sensitivity 0.95 (0.90–0.98), specificity 0.89 (0.81–0.95), PPV 0.93 (0.88–0.97), NPV 0.91 (0.83–0.96), and accuracy 92.5% (223/241 correctly classified).
- ML had the highest AUC; it was significantly higher than all time-based classifications but not significantly higher than ED visits or ICD-10 encounter-type coding. ML specificity was significantly higher than all other methods, and PPV was significantly higher than ICD-10 encounter type and all time-based classifications.
- ICD-10 encounter-type coding had sensitivity 1.00 and was described as superior for capturing all new injury encounters; ED visits were a reasonable proxy for pre-ICD-10 data. Time-based classification had no utility at <3 days, fair utility at 90 days, and good utility at >120 days.
- Agreement with manual review was highest for ML (K=0.84; 0.77–0.91), though not significantly higher than ICD-10 codes (0.73), ED visits (0.69), or >120 days (0.68). No significant accuracy differences by race or sex were found.
Methods
- Retrospective study using the St Louis region-wide hospital-based violence intervention program data repository (2010–2020). From 22 574 firearm-injury encounters, 500 patients with 808 encounters were randomly selected for manual review; final model dataset included 499 patients and 803 encounters, split 70/30 into training (562 encounters/337 patients) and testing (241 encounters/162 patients). ICD-10-specific evaluation used 151 encounters from 120 patients.
- Gold standard was manual chart review by two clinical providers, with discordant encounters adjudicated by a third reviewer; agreement was 753/808 (93.2%; Kappa=0.861).
- LASSO regression with 10-fold cross-validation (best lambda=0.008) used predictors including admission type, time between firearm injury visits, number of prior firearm injury ED visits, encounter type (ED or other), and diagnostic codes; 1745 covariates were reduced to 61 variables in the final model. Performance was compared with ED visits, time cutoffs (1, 3, 90, and optimal >120 days), and ICD-10 initial vs subsequent/sequela coding using AUC, sensitivity, specificity, PPV, NPV, accuracy, and Kappa.
Limitations
- The paper states that data were lacking for patients who sought care at non-trauma hospitals, non-acute care, or no care; it notes an estimated 30.5% seek care at non-trauma centers and that this could affect findings, particularly time-based approaches.
- The paper states limited generalizability to Native American/Alaska Native and Hispanic populations, and that non-Black and non-male sample sizes were small, so undetected bias may remain.
- The paper states that diagnostic-code models require ongoing evaluation because ICD codes evolve; it also notes the time-based assessment was limited to 36 subsequent encounters among 241 test encounters and variable cohort entry times. Evident from the text: the model was developed and tested in a single region and relied on manual review of a randomly selected subset rather than the full repository.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
19 of 29 cited sources checked · 10 supported · 1 flagged · accuracy not yet measured
- Premisecitation
It is estimated that 20% of patients with a firearm injury will be readmitted to the hospital within 90 days.
“In fact, it is estimated that 20% of patients with a firearm injury will be readmitted to the hospital within 90 days.3”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [3] Kalesan, 2019, Risk of 90-day readmission in patients after firearm injury hospitalization: a nationally representative retrospective cohort study, J Inj Violence Res source
The Results section reports a 10.5% absolute 90-day readmission rate among firearm-injury patients, not 20%. Its 20% figure describes the higher relative risk compared with pedestrians, not the proportion of firearm-injury patients readmitted. No page number is provided in our copy. (LLM, conf=0.93)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
- Premisecitation
It is estimated that 30.5% of patients seek care at a non-trauma center.
“It is estimated that 30.5% of patients seek care at a non-trauma center.31”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [31] Coupet, 2019, US Emergency Department encounters for firearm injuries according to presentation at trauma vs nontrauma centers, JAMA Surg source
- Findingown data
Of the 241 firearm injury encounters in the test dataset, 36 (14.9%) had more than one injury encounter.
“Of the 241 firearm injury encounters in the test dataset, 36 (14.9%) had more than one injury encounter.”
Quote in the paperSubstantial claim - Findingown data
The ML model had PPV of 0.93 (95% CI 0.88-0.97).
“given an AUC of 0.92 (0.88, 0.96), sensitivity of 0.95 (0.90, 0.98), specificity of 0.89 (0.81, 0.95), PPV of 0.93 (0.88, 0.97), and NPV of 0.91 (0.83, 0.96).”
Quote in the paperSubstantial claim - Findingown data
The ML model had an AUC of 0.92 (95% CI 0.88-0.96) in the test dataset.
“The ML model had excellent discrimination (0.92, 0.88-0.96)with high sensitivity (0.95, 0.90-0.98)and specificity (0.89, 0.81-0.95).”
- Also stated in: § Results · plain
“given an AUC of 0.92 (0.88, 0.96), sensitivity of 0.95 (0.90, 0.98), specificity of 0.89 (0.81, 0.95), PPV of 0.93 (0.88, 0.97), and NPV of 0.91 (0.83, 0.96).”
Quote in the paperSubstantial claim - Also stated in: § Results · plain
- Findingown data
The ML model's specificity was higher than all other methods.
“AUC was significantly higher than time-based outcomes, sensitivity was slightly (but not significantly) lower than other approaches, and specificity was higher than all other methods.”
Quote in the paperSubstantial claim - Findingown data
ICD-10 code encounter type had significantly lower PPV than ML.
“ED visits and ICD-10 code encounter type classification approaches were as, or more, sensitive classifiers than ML; however, they were both significantly less specific, and ICD-10 code encounter type had significantly lower PPV.”
Quote in the paperSubstantial claim - Findingown data
The ML model had sensitivity of 0.95 (95% CI 0.90-0.98).
“The ML model had excellent discrimination (0.92, 0.88-0.96)with high sensitivity (0.95, 0.90-0.98)and specificity (0.89, 0.81-0.95).”
- Also stated in: § Results · plain
“given an AUC of 0.92 (0.88, 0.96), sensitivity of 0.95 (0.90, 0.98), specificity of 0.89 (0.81, 0.95), PPV of 0.93 (0.88, 0.97), and NPV of 0.91 (0.83, 0.96).”
Quote in the paperSubstantial claim - Also stated in: § Results · plain
- Findingown data
ED visits and ICD-10 code encounter type classification approaches were significantly less specific than ML.
“ED visits and ICD-10 code encounter type classification approaches were as, or more, sensitive classifiers than ML; however, they were both significantly less specific, and ICD-10 code encounter type had significantly lower PPV.”
Quote in the paperSubstantial claim - Findingown data
The ML model's AUC was not significantly higher than ED visits or ICD-10 code encounter type classifications.
“The AUC for the ML model was significantly higher than any of the time-based classifications, but not significantly higher than ED visits or ICD-10 code encounter type classifications (Figure 2, Table 3).”
Quote in the paperSubstantial claim - Findingown data
There were no significant differences in classification accuracy by race or sex for any of the approaches evaluated.
“There were no significant differences in classification accuracy by race or sex for any of the approaches evaluated.”
Quote in the paperNot a substantial claim - Findingown data
The ML model's sensitivity was slightly but not significantly lower than other approaches.
“AUC was significantly higher than time-based outcomes, sensitivity was slightly (but not significantly) lower than other approaches, and specificity was higher than all other methods.”
Quote in the paperNot a substantial claim - Findingown data
ED visits and ICD-10 code encounter type classification approaches were as, or more, sensitive classifiers than ML.
“ED visits and ICD-10 code encounter type classification approaches were as, or more, sensitive classifiers than ML; however, they were both significantly less specific, and ICD-10 code encounter type had significantly lower PPV.”
Quote in the paperNot a substantial claim - Findingown data
The ML model had NPV of 0.91 (95% CI 0.83-0.96).
“given an AUC of 0.92 (0.88, 0.96), sensitivity of 0.95 (0.90, 0.98), specificity of 0.89 (0.81, 0.95), PPV of 0.93 (0.88, 0.97), and NPV of 0.91 (0.83, 0.96).”
Quote in the paperSubstantial claim - Findingown data
Compared to other classification approaches, the ML model had the highest AUC.
“Compared to other classification approaches, the ML had the highest AUC.”
Quote in the paperNot a substantial claim - Findingown data
For the 36 individuals with more than one injury encounter, the number of days between injuries had a median time of 36.0 days (range 4.0-1325.0, IQR 14.8-183.3).
“For those 36 individuals, the number of days between injuries had a median time of 36.0 days (range: 4.0-1325.0,IQR: 14.8-183.3).”
Quote in the paperSubstantial claim - Findingown data
The ML model's AUC was significantly higher than any of the time-based classifications.
“AUC was significantly higher than time-based outcomes, sensitivity was slightly (but not significantly) lower than other approaches, and specificity was higher than all other methods.”
- Also stated in: § Results · plain
“The AUC for the ML model was significantly higher than any of the time-based classifications, but not significantly higher than ED visits or ICD-10 code encounter type classifications (Figure 2, Table 3).”
Quote in the paperSubstantial claim - Also stated in: § Results · plain
- Findingown data
The ML model had an overall accuracy of 92.5% (223/241 correctly classified).
“with an overall accuracy of 92.5% (223/241 correctly classified).”
Quote in the paperSubstantial claim - Findingown data
The ML model had specificity of 0.89 (95% CI 0.81-0.95).
“The ML model had excellent discrimination (0.92, 0.88-0.96)with high sensitivity (0.95, 0.90-0.98)and specificity (0.89, 0.81-0.95).”
- Also stated in: § Results · plain
“given an AUC of 0.92 (0.88, 0.96), sensitivity of 0.95 (0.90, 0.98), specificity of 0.89 (0.81, 0.95), PPV of 0.93 (0.88, 0.97), and NPV of 0.91 (0.83, 0.96).”
Quote in the paperSubstantial claim - Also stated in: § Results · plain
- Findingown data
The ML model achieved the highest AUC among all approaches.
“The model achieved the highest AUC among all approaches, indicating its precision in identifying new firearm injury encounters.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingnone given
ED visits and ICD-10 visit type coding were essentially equivalent classifiers.
“ED visits and ICD-10 visit type coding were essentially equivalent classifiers”
Quote in the paperNot a substantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
Time-based classifications had no utility at less than 3 days.
“we found they had no utility at less than 3 days”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
ED visits were slightly more specific than ICD-10 codes.
“with ICD-10 codes being slightly more sensitive and ED visits slightly more specific.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
Time-based classifications had fair utility at 90 days.
“fair utility at 90 days”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
ICD-10 codes were slightly more sensitive than ED visits.
“with ICD-10 codes being slightly more sensitive and ED visits slightly more specific.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
Misclassification of firearm injury visit type was meaningfully higher when using ICD codes (12%) than compared to the ML model (7%).
“misclassification of firearm injury visit type was still meaningfully higher when using ICD codes (12%) than compared to our ML model (7%)”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
ICD-10 visit type codes had a sensitivity of 1.00.
“ICD-10 visit type codes had a sensitivity of 1.00”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
The ML model's specificity was significantly higher than all other methods.
“specificity was significantly higher than all other methods”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
Time-based classifications had good utility at more than 120 days.
“good utility at more than 120 days.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
The ML model did not have significantly higher AUC compared to classifications defined by ED visits or ICD-10 visit type.
“While the model did not have significantly higher AUC when compared to classifications defined by ED visits or ICD-10 visit type-specificity”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Findingown data
The ML model's PPV was significantly higher compared to ICD-10 visit type and all time-based classifications.
“PPV was significantly higher compared to ICD-10 visit type and all time-based classifications”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductnone given
A limitation of the study (and the majority of firearm injury research) is the lack of data on patients who sought care at a non-trauma hospital or non-acute care as well as those who did not seek medical care at all.
“a limitation of our study (and indeed the majority of firearm injury research) is the lack of data on patients who sought care at a non-trauma hospital or non-acute care as well as those who did not seek medical care at all.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Misclassification bias using ICD codes for encounter type is less than that of injury intent.
“The relatively lower misclassification in our study, compared to those of ICD injury intent (unintentional, intentional self-harm, assault, legal intervention, undetermined, or war and conflict) coding, indicates that misclassification bias using ICD codes for encounter type is less than that of injury intent.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductown data
Reviewers had a high level of agreement (753/808, 93.2%; Kappa = 0.861).
“Reviewers had a high level of agreement (753/808, 93.2%; Kappa ¼ 0.861).”
Quote in the paperSubstantial claim - Study design and conductown data
The varying amount of time in the cohort limited the ability to fully evaluate longer lengths of time.
“The varying amount of time in the cohort further limited our ability to fully evaluate longer lengths of time.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Interpretationreasoning
Some of the individuals included in the study may have had injuries that were not included in the dataset.
“Therefore, we expect that some of the individuals we included in our study may have had injuries that were not included in our dataset.”
Quote in the paperSubstantial claim - Study design and conductown data
The sample size in both non-Black and non-male populations were quite small.
“the sample size in both non-Black and non-male populations were quite small”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Premisenone given
There is a paucity of information on individuals who seek non-acute care.
“There is a paucity of information on individuals who seek non-acute care, and we were unable to find estimates of such.”
Quote in the paperNot a substantial claim - Study design and conductown data
Demographics for individuals in the training and test subsets were representative of the total population.
“Demographics for individuals in the training and test subsets were representative of the total population (Table 2).”
Quote in the paperNot a substantial claim - Interpretationreasoning
Inclusion of sex and race as covariates could over-estimate new firearm injury encounters in men and non-White populations.
“Given that men and those in non-White racial groups (except for Asian/Pacific Islander) are more likely to experience a firearm injury, inclusion of these factors as covariates could over-estimate new firearm injury encounters in these populations.”
Quote in the paperSubstantial claim- Rests on an assumption: There is no reason to believe that individuals from different sexes or races would differ clinically when presenting for a firearm injury.
- Study design and conductreasoning
Sex and race were not included as covariates in the ML model due to concerns about introducing algorithmic bias.
“We did not include sex and race in our ML model as covariates due to concerns it would introduce algorithmic bias.”
Quote in the paperSubstantial claim- Rests on an assumption: There is no reason to believe that individuals from different sexes or races would differ clinically when presenting for a firearm injury.
- Interpretationown data
Classification methods over-estimate new firearm injury encounters to varying degrees.
“Overall, our evaluation of classification methods indicates that these approaches over-estimate new firearm injury encounters to varying degrees.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductown data
The gold standard for new firearm injury encounter classification was manual chart review by two clinical providers, with discordant encounters adjudicated by a third member of the research team.
“We then randomly selected 500 patients (with 808 firearm injury encounters) for manual review by 2 clinical providers to provide the”
- Also stated in: § Participant and encounter selection · plain
“Each clinician (K.M. and M.S.) independently reviewed encounters and discordant encounters were adjudicated by a third member of the research team (V.M.).”
Quote in the paperSubstantial claim- Rests on an assumption: Manual chart review is a valid gold standard for classifying a firearm injury encounter as a new injury or a follow-up visit.
- Also stated in: § Participant and encounter selection · plain
- Interpretationown data
120 days may be a better cutoff value than 90 days when using time as a proxy measure to identify new firearm injury encounters.
“120 days may be a better cutoff value when using time as a proxy measure to identify new firearm injury encounters.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductnone given
The number of individuals who sought care from a non-trauma hospital is unknown in the study.
“Specifically for our study, the number of individuals who sought care from a non-trauma hospital is unknown.”
Quote in the paperSubstantial claim - Interpretationreasoning
There may have been undetected bias in the model.
“there may have been undetected bias in our model.”
Quote in the paperNot a substantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductown data
The dataset for assessing time-based classifications was limited to only 36 subsequent encounters of the 241 in the test data.
“Our dataset for assessing this, however, was limited to only 36 subsequent encounters of the 241 in the test data.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Interpretationreasoning
For datasets containing observations prior to ICD-10 implementation (October 2015), ED visits would be a reasonable proxy.
“For datasets containing observations prior to ICD-10 implementation (October 2015), ED visits would be a reasonable proxy.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductnone given
The number of individuals who did not seek medical care is also unknown in the study.
“The number of individuals who did not seek medical care is also unknown.”
Quote in the paperSubstantial claim - Interpretationown data
ML has the capability to accurately rule out follow-up visits.
“suggesting ML's capability in accurately ruling out follow-up visits, a key aspect in firearm injury prevention research.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Interpretationown data
ML is efficacious in classifying new firearm injury encounters.
“Our study demonstrates the efficacy of ML in classifying new firearm injury encounters.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Study design and conductown data
The study has a potential lack of generalizability to some of the populations heavily impacted by firearm injury, such as Native American/Alaska Native and Hispanic populations.
“Another limitation of our study is the potential lack of generalizability to some of the populations heavily impacted by firearm injury, such as Native American/Alaska Native and Hispanic populations.”
Quote in the paperNot a substantial claim - Interpretationown data
Advanced analytic techniques such as ML can improve classification of new firearm injury encounters compared to other approaches, particularly when ruling out follow-up encounters is key.
“In summary, advanced analytic techniques such as ML can improve classification of new firearm injury encounters compared to other approaches, particularly when ruling out follow-up encounters is key.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
- Classified as probably not a substantial claim. 20
- May combine two claims. 18
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).
PremisecitationThe FACTS Consortium stated a need for a greater focus on collecting longitudinal data that tests new and existing theoretical models to understand the epidemiology of firearm carriage, risky firearm behaviors, and firearm violence among adolescent and emerging adult populations.
“According to the Firearm Safety among Children and Teens (FACTS) Consortium (funded in 2017 by the National Institutes of Health), there is a need”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [16] Cunningham, 2019, Prevention of firearm injuries among children and adolescents: consensus-driven research agenda from the Firearm Safety Among Children and Teens (FACTS) consortium, JAMA Pediatr source
The Discussion’s Data Enhancement section calls for research guided by theoretical models and for longitudinal data collection to understand firearm behavior and causal relationships. The Epidemiology and Risk and Protective Factors section also discusses youth firearm carriage. But the paper frames its agenda around children and adolescents, not emerging adults, and does not specifically call for longitudinal tests of new and existing models of all three named outcomes. Page numbers were not available in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe time-based approach is most often applied to studies of firearm injury recurrence, with days since prior injury ranging from 1 to 90 days depending on the study.
“The time-based approach is most often applied to studies of firearm injury recurrence, with days since prior injury ranging from 1 to 90 days, depending on the study [7][8][9]”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [7] Kaufmann, 1998, A population-based study of trauma recidivism, J Trauma source
- cites [8] Rowhani-Rahbar, 2015, Firearm-related hospitalization and risk for subsequent violent injury, death, or crime perpetration: a cohort study, Ann Intern Med source
- cites [9] Carter, 2017, Individual and neighborhood characteristics of children seeking emergency department care for firearm injuries within the PECARN network, Acad Emerg Med source
The Methods (Data Analysis; page unavailable) use a time window to assess firearm-injury recidivism, supporting this paper as an example of a time-based approach. But the window is 12 months before and after the index visit, with returns within 72 hours excluded—not a 1-to-90-day window. This paper also does not establish how often the approach is used across studies. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
PremisecitationPrevious literature has highlighted discrepancies in diagnostic code validity concerning firearm injury encounter intent.
“Previous literature has highlighted discrepancies in diagnostic code validity concerning firearm injury encounter intent (unintentional, intentional self-harm, assault, legal intervention, undetermined, or war and conflict).4”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [4] Miller, 2022, Assessment of the accuracy of firearm injury intent coding at 3 US hospitals, JAMA Netw Open source
Probably not a substantial claim: may not pass S2 (checkable).
PremisecitationAdvanced computational techniques, such as ML and natural language processing, are promising ways to address diagnostic classification problems where issues with ICD coding persist.
“Advanced computational techniques, such as ML and natural language processing (NLP), are promising ways to address diagnostic classification problems where issues with ICD coding persist.11,15”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [11] Pear, 2020, Risk factors for assaultive reinjury and death following a nonfatal firearm assault injury: a population-based retrospective cohort study, Prev Med source
- cites [15] MacPhaul, 2023, Classifying firearm injury intent in electronic hospital records using natural language processing, JAMA Netw Open source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe homicide death ratio affiliated with firearms is 5 times higher in men than in women.
“The homicide death ratio affiliated with firearms is 5 times higher in men than in women.22”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [22] Young, 2022, US racial and sex-based disparities in firearm-related death trends from 1981-2020, PLoS One source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study followed the TRIPOD checklist in this report.
“We followed the TRIPOD checklist in this report.18”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [18] Collins, 2015, Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement, Br J Surg source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationTwo proxy measures commonly used in the existing literature are ED visits and time between firearm injury encounters.
“Two proxy measures commonly used in the existing body of literature are (1) ED visits and (2) time between firearm injury encounters.2,4-9”
Quote in the paperSubstantial claim5 of 7 sources checked- cites [2] Kaufman, 2021, Epidemiologic trends in fatal and nonfatal firearm injuries in the US, 2009-2017, JAMA Intern Med source
The Abstract and the section on data sources show that this paper uses emergency department visits to estimate nonfatal firearm injuries and describes earlier work using ED data. The passages do not identify time between firearm injury encounters as a measure used in the literature; searches for that phrase and for proxy measures found no such discussion. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [4] Miller, 2022, Assessment of the accuracy of firearm injury intent coding at 3 US hospitals, JAMA Netw Open source
The Introduction (page not available) describes hospital data on patients presenting to emergency departments as a major source of information about nonfatal firearm injuries, supporting the ED-visit part in a broad sense. The paper does not identify ED visits as a commonly used proxy measure or discuss time between firearm injury encounters as a measure. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [5] Miller, 2024, Costs of fatal and nonfatal firearm injuries in the U.S., 2019 and 2020, Am J Prev Med source
The Discussion (page unavailable) says most current cost studies use emergency-department and inpatient datasets, supporting the narrower point that ED-based data are commonly used in that literature. It does not describe ED visits as a proxy measure or identify time between firearm-injury encounters as a commonly used measure; searches for those terms found no supporting passage. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [6] Schnippel, 2021, Nonfatal firearm injuries by intent in the United States: 2016-2018 hospital discharge records from the healthcare cost and utilization project, West J Emerg Med source
The Discussion describes prior analyses based on emergency-department visits, supporting ED visits as a measure used in the literature. It also mentions 30- and 90-day readmissions, but does not identify time between firearm injury encounters as a commonly used proxy measure. The Methods and Results count injury episodes rather than measuring intervals between them. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [7] Kaufmann, 1998, A population-based study of trauma recidivism, J Trauma source
- cites [8] Rowhani-Rahbar, 2015, Firearm-related hospitalization and risk for subsequent violent injury, death, or crime perpetration: a cohort study, Ann Intern Med source
- cites [9] Carter, 2017, Individual and neighborhood characteristics of children seeking emergency department care for firearm injuries within the PECARN network, Acad Emerg Med source
The Methods (Data Analysis; page unavailable) identify repeat ED visits within 12 months of an index firearm injury, and the Results (Emergency Department Outcomes and Recidivism; page unavailable) report subsequent visits and firearm injuries. This supports ED visits as a measure, but the paper does not report time between separate firearm injury encounters as a measured interval; its 12-month follow-up window is not that measure. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationPeople in the Black racial group have a firearm-related homicide mortality rate 7 times higher than those in the White racial group and 5 times higher than those in the American Indian/Alaska Native racial group.
“People in the Black racial group have a firearm-related homicide mortality rate 7 times higher than those in the White racial group and 5 times higher than those in the American Indian/Alaska Native racial group (race with second highest rate).22”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [22] Young, 2022, US racial and sex-based disparities in firearm-related death trends from 1981-2020, PLoS One source
May combine two claims.
PremisecitationFirearm injury disproportionately affects males and Black individuals.
“Firearm injury disproportionately affects males and black individuals.”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [22] Young, 2022, US racial and sex-based disparities in firearm-related death trends from 1981-2020, PLoS One source
The Results report substantially higher firearm homicide and suicide death rates for men than women, and higher firearm homicide death rates for Black people than White people. But the paper studies deaths, not firearm injury generally; it also reports that White people had the highest firearm suicide death rate. (LLM, conf=0.83)
Experimental — accuracy not yet measured - cites [5] Miller, 2024, Costs of fatal and nonfatal firearm injuries in the U.S., 2019 and 2020, Am J Prev Med source
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThese findings meaningfully impact the rigor of clinical research designed to evaluate the epidemiology of new and recurrent firearm injury encounters across populations as well as re-injury evaluations of violence and injury intervention programs.
“These findings also meaningfully impact the rigor of clinical research designed to evaluate the epidemiology of new and recurrent firearm injury encounters across populations as well as re-injury evaluations of violence and injury intervention programs (eg, hospital-based violence intervention programs).12”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [12] Mueller, 2023, Incidence of and factors associated with recurrent firearm injury among patients presenting to St Louis trauma centers, 2010 to 2019: a cohort study, Ann Intern Med source
- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationML has been used to classify new firearm injury encounters by our team and by one other published study.
“Machine learning (ML) has also been used to classify new firearm injury encounters by our team, and we are aware of one other published study that has also used ML to classify new firearm injury encounters.11,12”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [11] Pear, 2020, Risk factors for assaultive reinjury and death following a nonfatal firearm assault injury: a population-based retrospective cohort study, Prev Med source
- cites [12] Mueller, 2023, Incidence of and factors associated with recurrent firearm injury among patients presenting to St Louis trauma centers, 2010 to 2019: a cohort study, Ann Intern Med source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTrends are similar for nonfatal firearm injuries.
“Trends are similar for nonfatal firearm injuries as well.5”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [5] Miller, 2024, Costs of fatal and nonfatal firearm injuries in the U.S., 2019 and 2020, Am J Prev Med source
May combine two claims.
Findingown dataAgreement (Kappa) was highest for the ML model (0.84; 95% CI 0.77-0.91), but not significantly higher than ICD-10 codes (0.73; 0.62-0.84), ED visits (0.69; 0.60-0.79), or encounters after 120 days (0.68; 0.59-0.78).
“Agreement (K) was also highest for the ML model (0.84; 0.77, 0.91), but not significantly higher than ICD-10 codes (0.73; 0.62, 0.84), ED visits (0.69; 0.60, 0.79), or encounters after 120 days (0.68; 0.59, 0.78).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataFor time-based classifications, sensitivity remained constant while specificity and AUC increased as the number of days increased.
“Sensitivity for time-based classifications remained constant, with increasing specificity and thus increasing AUC, as the number of days increased (Table 3).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe ML model predicted 153/241 (63.5%) encounters were acute firearm injuries, compared to 151/241 (62.7%) classified by the gold standard.
“The ML model predicted 153/241 (63.5%) encounters were acute firearm injuries, compared to 151/241 (62.7%) classified by the gold standard, with an overall accuracy of 92.5% (223/241 correctly classified).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe distribution of days between injuries was right-skewed, with the majority of values (25, 69.4%) less than or equal to 120 days.
“The distribution was rightskewed, with the majority of values (25, 69.4%) less than or equal to 120 days.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe minimum squared Euclidian distance between sensitivity and specificity was 0.97 at 121 days, and greater than 120 days was selected as the optimal cut-off value for time between firearm injury encounters.
“The minimum Euclidian distance between sensitivity and specificity squared was 0.97 (range: 0.97-1.61),at 121 days.Therefore, we selected greater than 120 days as our optimal cut-off value for time between firearm injury encounters as a classification approach.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningAny classification model using diagnostic codes necessitates ongoing model evaluation and, potentially, modification.
“This means that any classification model using diagnostic codes necessitates ongoing model evaluation and, potentially, modification.”
Quote in the paperNot a substantial claim- Rests on an assumption: ICD-10 codes are updated regularly to include new diagnoses.
May combine two claims.
Study design and conductcitationThe model's effect size (AUC converted to Cohen's D of 2.0) and accuracy (93%) were both above sufficient sample size thresholds (0.50 and 80%, respectively).
“Considering the sample size criteria proposed by Rajput et al., the model's effect size (AUC converted to a Cohen's D of 2.0) and its accuracy (93%) were both well above sufficient sample size thresholds (0.50 and 80%, respectively).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rajput et al.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA study from 2000 surveyed 93 incarcerated men and found that 8% of them did not seek medical care for their last firearm injury.
“we did find a study from 2000 that surveyed 93 incarcerated men and found that 8% of them did not seek medical care for their last firearm injury.32”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [32] May, 2000, Medical care solicitation by criminals with gunshot wound injuries: a survey of Washington, DC, jail detainees, J Trauma source
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenLASSO regression was used to develop the model, which was cross-validated (k = 10) to identify the best lambda to tune the final model to avoid overfitting.
“We used LASSO regression to develop the model, which was then cross-validated (k ¼ 10) to identify the best lambda to tune the final model to avoid overfitting.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenA limitation of the model (or any model built using diagnostic codes) is the evolving nature of ICD codes since ICD-10 codes are updated regularly to include new diagnoses.
“a limitation of our model (or any model built using diagnostic codes) is the evolving nature of ICD codes since ICD-10 codes are updated regularly to include new diagnoses.”
Quote in the paperNot a substantial claim- Rests on an assumption: ICD-10 codes are updated regularly to include new diagnoses.
May combine two claims.
Interpretationown dataML can be used to identify new firearm injury encounters and may be particularly useful in studies assessing re-injuries.
“ML can be used to identify new firearm injury encounters and may be particularly useful in studies assessing re-injuries.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe ML model performed well and appeared well-balanced in both capturing new firearm injury encounters and ruling out follow-up encounters for prior injuries.
“The model performed well and appears to be well-balanced in both capturing new firearm injury encounters and ruling out follow-up encounters for prior injuries given an AUC of 0.92 (0.88, 0.96), sensitivity of 0.95 (0.90, 0.98), specificity of 0.89 (0.81, 0.95), PPV of 0.93 (0.88, 0.97), and NPV of 0.91 (0.83, 0.96).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenMissing injury severity scores were coded as zero and missing admission type was added as an additional category for that variable.
“Missing injury severity scores were coded as zero and missing admission type was added as an additional category for that variable.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe study tested for differences in accuracy for sex and race.
“We did test for differences in accuracy for sex and race”
Quote in the paperNot a substantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
May combine two claims.
Interpretationown dataOther approaches to classifying new firearm injury encounters likely yield over-estimates, particularly if they use a time-based cut-off less than 90 days, and to a lesser extent if using ED visits as a proxy measure or ICD-10 encounter type.
“Other approaches to classifying new firearm injury encounters likely yield over-estimates, particularly if they use a time-based cut-off less than 90 days, and to a lesser extent if using ED visits as a proxy measure or ICD-10 encounter type.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationcitationThe results of the primary outcome align with the work of Pear et al., who also reported a high AUC using an ML model for classifying firearm injury visit type.
“The results of our primary outcome align with the work of Pear et al., who also reported a high AUC using an ML model for classifying firearm injury visit type.11”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [11] Pear, 2020, Risk factors for assaultive reinjury and death following a nonfatal firearm assault injury: a population-based retrospective cohort study, Prev Med source
- Rests on an assumption: Manual review correctly identifies new firearm injury encounters.
May combine two claims.
Interpretationown dataML successfully delineated new firearm injury encounters and outperformed other approaches in ruling out encounters for follow-up.
“ML successfully delineated new firearm injury encounters, outperforming other approaches in ruling out encounters for follow-up.”
Quote in the paperSubstantial claim- Rests on an assumption: Manual chart review is a valid gold standard for classifying a firearm injury encounter as a new injury or a follow-up visit.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Study design and conductnone givenThe sample in the study is representative of its region, which has a limited number of individuals in these populations.
“The sample in our study is representative of its region, which has a limited number of individuals in these populations.”
Quote in the paperNot a substantial claim- Rests on an assumption: The sample in the study is representative of its region.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningThis could have impacted the findings, particularly with time-based approaches.
“This could have impacted our findings, particularly with time-based approaches.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationThe precision of classification methods remains unestablished and comparisons between these methods have not been published.
“While the variability of classification methods is known, and was recently (2023) described in a systematic review on firearm injury recurrence, the precision of these methods remain unestablished and comparisons between these methods, to our knowledge, have not been published.13”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13] Shayan, 2023, A systematic review of recurrent firearm injury rates in the United States, Prev Med source
The paper relies on this without support.
Those who survive firearm injuries are at high risk of recurrent firearm injury.
“Those who survive firearm injuries are a key target for secondary prevention intervention strategies (eg, hospital-based violence intervention programs) due to their high risk of recurrent firearm injury.”
The paper relies on this without support.
Firearm injuries are often severe and routinely necessitate multiple healthcare encounters as part of post-acute care or to treat complications and new conditions that are sequela of the initial injury.
“Firearm injuries are often severe, routinely necessitating multiple healthcare encounters as part of routine post-acute care or to treat complications and new conditions that were sequela of the initial injury.”
The paper relies on this without support.
Follow-up encounters for firearm injuries obfuscate the true incidence of new firearm injury encounters and assessments of firearm injury recurrence in hospital data due to diagnostic coding limitations.
“Due to diagnostic coding limitations, these follow-up encounters for firearm injuries obfuscate the true incidence of new firearm injury encounters and assessments of firearm injury recurrence in hospital data.”
The paper relies on this without support.
There is no reason to believe that individuals from different sexes or races would differ clinically when presenting for a firearm injury.
“There is also no reason to believe that individuals from different sexes or races would differ clinically when presenting for a firearm injury.”
The paper relies on this without support.
Manual chart review is a valid gold standard for classifying a firearm injury encounter as a new injury or a follow-up visit.
“Our gold standard for new firearm injury encounter classification was manual chart review.”
“The gold standard for the outcome variable, classification of a firearm injury encounter as new or follow-up was defined by manual review (described above in the participant and encounter selection section).”
The paper relies on this without support.
Coding limitations persist across healthcare settings, potentially leading to misclassification.
“Yet coding limitations persist across healthcare settings, potentially leading to misclassification.”
The paper relies on this without support.
ICD-10 enhanced coding to specify whether an encounter was initial, subsequent, or sequela, whereas ICD-9 lacked specificity regarding encounter type.
“ICD-9 lacked specificity regarding encounter type; whereas ICD-10 enhanced coding to specify if the encounter was initial, subsequent (part of recovery care), or sequela (complications or conditions that arise from a prior injury).”
The paper relies on this without support.
Manual review correctly identifies new firearm injury encounters.
“Classification of new firearm injury encounters, comparing all methods to manual review.”
The paper relies on this without support.
ICD-10 codes are updated regularly to include new diagnoses.
“ICD-10 codes are updated regularly to include new diagnoses.”
The paper relies on this without support.
The sample in the study is representative of its region.
“The sample in our study is representative of its region”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Dual Medicaid/Medicare subgroup counts do not add up
MinorThe training and testing counts sum to 5, but the manual-review total is 9.
Reported 9 · recomputed 5
“| Dual Medicaid/Medicare | 260 (1.9%) | 9 (1.8%) | 4 (1.2%) | 1 (0.6%) |”
Medicare subgroup counts exceed the manual-review total
MinorThe training and testing counts sum to 9, but the manual-review total is 5.
Reported 5 · recomputed 9
“| Medicare | 214 (1.6%) | 5 (1.0%) | 8 (2.4%) | 1 (0.6%) |”
Cites
Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., et al. (2019) · Journal of Open Source Software · doi:10.21105/joss.01686 · Cited by 22,138 · see citing works
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927 · see citing works
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697 · see citing works
ROCR: visualizing classifier performance in R
Sing, T., Sander, O., Beerenwinkel, N., & Lengauer, T. (2005) · Computer applications in the biosciences · doi:10.1093/bioinformatics/bti623 · Cited by 3,172 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Surgery · doi:10.1002/bjs.9736 · Cited by 889 · see citing works
Evaluation of a decided sample size in machine learning applications
Rajput, D., Wang, W.-J., & Chen, C.-C. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05156-9 · Cited by 411 · see citing works
Epidemiologic trends in fatal and nonfatal firearm injuries in the US, 2009-2017.
Kaufman, E. J., Wiebe, D. J., Xiong, R. A., Morrison, C. N., Seamon, M. J., & Delgado, M. K. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.6696 · Cited by 285 · see citing works
Rowhani-Rahbar, A., Zatzick, D., Wang, J., Mills, B. M., Simonetti, J. A., Fan, M. D., & Rivara, F. P. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2362 · Cited by 150 · see citing works
Carter, P. M., Cook, L. J., Macy, M. L., Zonfrillo, M. R., Stanley, R. M., Chamberlain, J. M., Fein, J. A., Alpern, E. R., Cunningham, R. M., & the Pediatric Emergency Care Applied Research Network (PECARN) (2017) · Academic Emergency Medicine · doi:10.1111/acem.13200 · Cited by 95 · see citing works
A Population-Based Study of Trauma Recidivism
Kaufmann, C. R., Branas, C. C., & Brawley, M. L. (1998) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/00005373-199808000-00019 · Cited by 88 · see citing works
Cunningham, R. M., Carter, P. M., Ranney, M. L., Walton, M., Zeoli, A. M., Alpern, E. R., Branas, C., Beidas, R. S., Ehrlich, P. F., Goyal, M. K., et al. (2019) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2019.1494 · Cited by 86 · see citing works
Costs of fatal and nonfatal firearm injuries in the U.S., 2019 and 2020
Miller, G. F., Barnett, S. B. L., Florence, C. S., McDavid Harrison, K., Dahlberg, L. L., & Mercy, J. A. (2024) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2023.09.026 · Cited by 76 · see citing works
Assessment of the accuracy of firearm injury intent coding at 3 US hospitals
Miller, M., Azrael, D., Yenduri, R., Barber, C., Bowen, A., MacPhaul, E., Mooney, S. J., Zhou, L., Goralnick, E., & Rowhani-Rahbar, A. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.46429 · Cited by 52 · see citing works
Coupet, E., Huang, Y., & Delgado, M. K. (2019) · JAMA Surgery · doi:10.1001/jamasurg.2018.4640 · Cited by 42 · see citing works
Pear, V. A., McCort, C. D., Kravitz-Wirtz, N., Shev, A. B., Rowhani-Rahbar, A., & Wintemute, G. J. (2020) · Preventive Medicine · doi:10.1016/j.ypmed.2020.106198 · Cited by 42 · see citing works
Schnippel, K., Burd-Sharps, S., Miller, T., Lawrence, B., & Swedler, D. (2021) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2021.3.51925 · Cited by 40 · see citing works
(2019) · Journal of Injury and Violence Research · doi:10.5249/jivr.v11i1.979 · Cited by 37 · see citing works
Mueller, K. L., Trolard, A., Moran, V., Landman, J. M., & Foraker, R. (2021) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2020.100683 · Cited by 32 · see citing works
Self-citationDonnelly, K. A., Badolato, G. M., & Goyal, M. K. (2020) · Pediatric Emergency Care · doi:10.1097/pec.0000000000002272 · Cited by 24 · see citing works
US racial and sex-based disparities in firearm-related death trends from 1981-2020
Young, L. J., & Xiang, H. (2022) · PLOS ONE · doi:10.1371/journal.pone.0278304 · Cited by 22 · see citing works
Mueller, K. L., Cooper, B. P., Moran, V., Lew, D., Ancona, R., Landman, J. M., Spruce, M., Marotta, P., Liss, D. B., Mancini, M. A., et al. (2023) · Annals of Internal Medicine · doi:10.7326/m23-0069 · Cited by 21 · cited 3× · see citing works
“12 It is important to note, however, that a limitation of our study (and indeed the majority of firearm injury research) is the lack of data on patients who sought care at a non-trauma hospital or non-acute care as well as those who did not seek medical care at all.”
BackgroundResultSelf-citationClassifying firearm injury intent in electronic hospital records using natural language processing
MacPhaul, E., Zhou, L., Mooney, S. J., Azrael, D., Bowen, A., Rowhani-Rahbar, A., Yenduri, R., Barber, C., Goralnick, E., & Miller, M. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.5870 · Cited by 17 · see citing works
Carter, P. M., Zimmerman, M. A., & Cunningham, R. M. (2021) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2021.1913741 · Cited by 15 · see citing works
Byrne, D. W., Domenico, H. J., & Moore, R. P. (2024) · Korean Journal of Radiology · doi:10.3348/kjr.2023.1016 · Cited by 13 · see citing works
May, J. P., Hemenway, D., Oen, R., & Pitts, K. R. (2000) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/00005373-200001000-00023 · Cited by 9 · see citing works
A systematic review of recurrent firearm injury rates in the United States
Shayan, M., Lew, D., Mancini, M., Foraker, R. E., Doering, M., & Mueller, K. L. (2023) · Preventive Medicine · doi:10.1016/j.ypmed.2023.107443 · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Byrne, 2024, Artificial Intelligence for Improved Patient Outcomes
Cited by
Sorted by citations
Evaluation of a Region-Wide Hospital-Based Violence Intervention Program
Mueller, K. L., Kaser, T., Cooper, B. P., Lew, D., Moran, V. L., Ancona, R. M., Chapman-Kramer, K., Coffey, M., Page, K., Batha, A., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-01576 · Cited by 7 · see citing works
Self-citationFactors Associated With Recurrent Pediatric Firearm Injury
Miller, Z. M., Cooper, B. P., Lew, D., Ancona, R. M., Moran, V. L., Behr, C. A., Spruce, M. W., Kranker, L. M., Mancini, M. A., Vogel, M., et al. (2024) · Annals of Internal Medicine · doi:10.7326/m24-0430 · Cited by 4 · see citing works
Self-citationElkarous, L., Jeridi, M. H., & Dhouibi, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-27949-z · Cited by 1 · see citing works
Firearm injury risk detection and prevention
Henry, S. B. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae245 · Cited by 1 · see citing works
Spruce, M. W., Freeman, M. N., Kranker, L. M., Moran, V. L., Behr, C. A., Lew, D., Cooper, B. P., & Mueller, K. L. (2026) · JEM Reports · doi:10.1016/j.jemrpt.2026.100254
Self-citationInitial Injury Type and Violent Reinjury
Szoko, N., Lew, D., Kaser, T., Miller, Z. M., Vogel, M., Mancini, M. A., Kranker, L. M., Behr, C. A., Moran, V. L., Ancona, R. M., et al. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.18199
Self-citationThe Crisis of Adolescent Firearm Reinjury: A 14-Year Retrospective Cohort Analysis
Mueller, K. L., Lew, D., Szoko, N., Miller, Z. M., Kaser, T., Cooper, B. P., Behr, C. A., Clukies, L., Finney, J. D., Spruce, M. W., et al. (2026) · Hospital Pediatrics · doi:10.1542/hpeds.2026-009207
Self-citation
- Authors
- Ancona, R. M., Cooper, B. P., Foraker, R., Kaser, T., Adeoye, O., & Mueller, K. L.
- Affiliations
- Washington University in St. Louis (US)
- Corresponding author
- Ancona, Rachel M
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-07-08
- Volume
- 31
- Issue
- 10
- Pages
- 2165-2172
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC11413423
- OpenAlex
- W4400409593
- Semantic Scholar
- 1198a525f7bac2345051a5f59ac963cdc3d1461a
- Cited by
- 7
- References
- 26
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 2
- FWCI
- 3.03
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (submittedVersion)
- PubMed Central (submittedVersion)
- Funders
- National Institutes of Health
- Eunice Kennedy Shriver National Institute of Child Health & Human Development (K23HD107520)
- Topics
- Social Sciences › Health › Gun Ownership and Violence Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- firearm injuries, LASSO, violence intervention programs, injury recurrence, emergency department visits, time since injury, diagnosis codes, St. Louis
- MeSH terms
- Machine Learning, Adult, Emergency Service, Hospital, Female, Firearms, Humans, Male, Missouri, Retrospective Studies, Wounds, Gunshot
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-27.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-27.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
