Improving Radiographic Fracture Recognition Performance and Efficiency Using Artificial Intelligence
Open accessAbstract
Background Missed fractures are a common cause of diagnostic discrepancy between initial radiographic interpretation and the final read by board-certified radiologists. Purpose To assess the effect of assistance by artificial intelligence (AI) on diagnostic performances of physicians for fractures on radiographs. Materials and Methods This retrospective diagnostic study used the multi-reader, multi-case methodology based on an external multicenter data set of 480 examinations with at least 60 examinations per body region (foot and ankle, knee and leg, hip and pelvis, hand and wrist, elbow and arm, shoulder and clavicle, rib cage, and thoracolumbar spine) between July 2020 and January 2021. Fracture prevalence was set at 50%. The ground truth was determined by two musculoskeletal radiologists, with discrepancies solved by a third. Twenty-four readers (radiologists, orthopedists, emergency physicians, physician assistants, rheumatologists, family physicians) were presented the whole validation data set (n = 480), with and without AI assistance, with a 1-month minimum washout period. The primary analysis had to demonstrate superiority of sensitivity per patient and the noninferiority of specificity per patient at –3% margin with AI aid. Stand-alone AI performance was also assessed using receiver operating characteristic curves. Results A total of 480 patients were included (mean age, 59 years ± 16 [standard deviation]; 327 women). The sensitivity per patient was 10.4% higher (95% CI: 6.9, 13.9; P < .001 for superiority) with AI aid (4331 of 5760 readings, 75.2%) than without AI (3732 of 5760 readings, 64.8%). The specificity per patient with AI aid (5504 of 5760 readings, 95.6%) was noninferior to that without AI aid (5217 of 5760 readings, 90.6%), with a difference of +5.0% (95% CI: +2.0, +8.0; P = .001 for noninferiority). AI shortened the average reading time by 6.3 seconds per examination (95% CI: –12.5, –0.1; P = .046). The sensitivity by patient gain was significant in all regions (+8.0% to +16.2%; P < .05) but shoulder and clavicle and spine (+4.2% and +2.6%; P = .12 and .52). Conclusion AI assistance improved the sensitivity and may even improve the specificity of fracture detection by radiologists and nonradiologists, without lengthening reading time. Published under a CC BY 4.0 license. Online supplemental material is available for this article. See also the editorial by Link and Pedoia in this issue.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Fatigue among Clinicians and the Safety of Patients
Gaba, D. M., & Howard, S. K. (2002) · New England Journal of Medicine · doi:10.1056/nejmsa020846 · Cited by 680 · see citing works
Deep neural network improves fracture detection by clinicians
Lindsey, R., Daluiski, A., Chopra, S., Lachapelle, A. S., Mozer, M. C., Sicular, S., Hanel, D. P., Gardner, M. J., Gupta, A., Hotchkiss, R. J., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806905115 · Cited by 654 · see citing works
Kim, D. H., & MacKinnon, T. (2017) · Clinical Radiology · doi:10.1016/j.crad.2017.11.015 · Cited by 504 · see citing works
Cognitive and System Factors Contributing to Diagnostic Errors in Radiology
Lee, C. S., Nagy, P. G., Weaver, S. J., & Newman-Toker, D. E. (2013) · American Journal of Roentgenology · doi:10.2214/ajr.12.10375 · Cited by 415 · see citing works
Cheng, C., Ho, T., Lee, T.-Y., Chang, C.-C., Chou, C.-C., Chen, C.-C., Chung, I., & Liao, C. (2019) · European Radiology · doi:10.1007/s00330-019-06167-y · Cited by 311 · see citing works
Dembrower, K., Wåhlin, E., Liu, Y., Salim, M., Smith, K., Lindholm, P., Eklund, M., & Strand, F. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30185-0 · Cited by 305 · see citing works
Long Radiology Workdays Reduce Detection and Accommodation Accuracy
Krupinski, E. A., Berbaum, K. S., Caldwell, R. T., Schartz, K. M., & Kim, J. (2010) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2010.03.004 · Cited by 276 · see citing works
Egglin, T. K. (1996) · JAMA · doi:10.1001/jama.1996.03540210060035 · Cited by 192 · see citing works
Duron, L., Ducarouge, A., Gillibert, A., Lainé, J., Allouche, C., Cherel, N., Zhang, Z., Nitche, N., Lacave, E., Pourchot, A., et al. (2021) · Radiology · doi:10.1148/radiol.2021203886 · Cited by 164 · see citing works
Self-citationAssessment of a deep-learning system for fracture detection in musculoskeletal radiographs
Jones, R. M., Sharma, A. K., Hotchkiss, R. J., Sperling, J. W., Hamburger, J., Ledig, C., O’Toole, R. V., Gardner, M. J., Venkatesh, S., Roberts, M. M., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00352-w · Cited by 139 · see citing works
The effect of clinical information on radiology reporting: A systematic review
Castillo, C., Steffens, T., Sim, L., & Caffery, L. J. (2020) · Journal of Medical Radiation Sciences · doi:10.1002/jmrs.424 · Cited by 130 · see citing works
A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs
Cheng, C., Wang, Y., Chen, H., Hsiao, P.-M., Yeh, C., Hsieh, C., Miao, S., Xiao, J. Z., Liao, C., & Lü, L. (2021) · Nature Communications · doi:10.1038/s41467-021-21311-3 · Cited by 101 · see citing works
The Insidious Problem of Fatigue in Medical Imaging Practice
Reiner, B. I., & Krupinski, E. (2012) · Journal of Digital Imaging · doi:10.1007/s10278-011-9436-4 · Cited by 88 · see citing works
Chen, H., Hsu, B. W., Yin, Y.-K., Lin, F., Yang, T.-H., Yang, R., Lee, C., & Tseng, V. S. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0245992 · Cited by 68 · see citing works
Fernholm, R., Härenstam, K. P., Wachtler, C., Nilsson, G. H., Holzmann, M. J., & Carlsson, A. C. (2019) · European Journal of General Practice · doi:10.1080/13814788.2019.1625886 · Cited by 65 · see citing works
Review of the Basics of Cognitive Error in Emergency Medicine: Still No Easy Answers
Hartigan, S., Brooks, M. N., Hartley, S. E., Forrest, R., Santen, S. A., & Hemphill, R. R. (2020) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2020.7.47832 · Cited by 56 · see citing works
Horstink, L., Langeraar, J. J., Mauritz, G.-J., et al. (2020) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-020-00727-8 · Cited by 46 · see citing works
Tobler, P., Cyriac, J., Kovacs, B. K., et al. (2021) · European Radiology · doi:10.1007/s00330-021-07811-2 · Cited by 43 · see citing works
Ren, M., & Yi, P. H. (2021) · Skeletal Radiology · doi:10.1007/s00256-021-03739-2 · Cited by 30 · see citing works
Missed foot fractures in multiple trauma patients
Fitschen‐Oestern, S., Lippross, S., Lefering, R. L., et al. (2019) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-019-2501-8 · Cited by 29 · see citing works
Fitschen‐Oestern, S., Lippross, S., Lefering, R. L., Klüter, T., Behrendt, P., Weuster, M., Seekamp, A., & Dgu, T. (2020) · Injury · doi:10.1016/j.injury.2020.04.022 · Cited by 24 · see citing works
Etiology and disposition associated with radiology discrepancies on emergency department patients
Gergenti, L., & Olympia, R. P. (2019) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2019.02.027 · Cited by 18 · see citing works
Critical evaluation of deep neural networks for wrist fracture detection
Raisuddin, A. M., Vaattovaara, E., Nevalainen, M. T., Nikki, M. J., Järvenpää, E., Makkonen, K., Pinola, P., Palsio, T., Niemensivu, A., Tervonen, O. A., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-85570-2 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Agrawal, A., Khatri, G. D., Khurana, B., Sodickson, A. D., Liang, Y., & Dreizin, D. (2023) · Emergency Radiology · doi:10.1007/s10140-023-02121-0 · Cited by 41 · see citing works
- Authors
- Guermazi, A., Tannoury, C., Kompel, A. J., Murakami, A. M., Ducarouge, A., Gillibert, A., Li, X., Tournier, A., Lahoud, Y., Jarraya, M., Lacave, E., Rahimi, H., Pourchot, A., Parisien, R. L., Merritt, A. C., Comeau, D., Regnard, N.-E., & Hayashi, D.
- Affiliations
- Boston University (US)
- Boston College (US)
- ORCID
- Venue
- Radiology
- Published
- 2022-03
- Volume
- 302
- Issue
- 3
- Pages
- 627-636
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W4200013359
- Semantic Scholar
- 9db096123d72238b66bf262d226f41c551bef3a6
- Cited by
- 307
- References
- 23
- Citations by year
- 2026: 71 · 2025: 104 · 2024: 72 · 2023: 36 · 2022: 22 · 2021: 1
- FWCI
- 7.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Radiology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (Le Centre pour la Communication Scientifique Directe) (submittedVersion)
- Funders
- AI and software and built the study sample and design
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- fracture detection, artificial intelligence, radiographic interpretation, missed fractures, diagnostic accuracy, sensitivity and specificity, AI-assisted diagnosis, musculoskeletal radiology, reading time reduction, non-inferiority, multi-reader multi-case study, emergency physicians, physician assistants, thoracolumbar fractures
- MeSH terms
- Artificial Intelligence, Diagnostic Errors, Female, Humans, Male, Middle Aged, Radiographic Image Interpretation, Computer-Assisted, Retrospective Studies, Sensitivity and Specificity, Fractures, Bone, Quality Improvement, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
