Deep Learning–Assisted Diagnosis of Cerebral Aneurysms Using the HeadXNet Model
Open accessAbstract
Importance: Deep learning has the potential to augment clinician performance in medical imaging interpretation and reduce time to diagnosis through automated segmentation. Few studies to date have explored this topic. Objective: To develop and apply a neural network segmentation model (the HeadXNet model) capable of generating precise voxel-by-voxel predictions of intracranial aneurysms on head computed tomographic angiography (CTA) imaging to augment clinicians' intracranial aneurysm diagnostic performance. Design, Setting, and Participants: In this diagnostic study, a 3-dimensional convolutional neural network architecture was developed using a training set of 611 head CTA examinations to generate aneurysm segmentations. Segmentation outputs from this support model on a test set of 115 examinations were provided to clinicians. Between August 13, 2018, and October 4, 2018, 8 clinicians diagnosed the presence of aneurysm on the test set, both with and without model augmentation, in a crossover design using randomized order and a 14-day washout period. Head and neck examinations performed between January 3, 2003, and May 31, 2017, at a single academic medical center were used to train, validate, and test the model. Examinations positive for aneurysm had at least 1 clinically significant, nonruptured intracranial aneurysm. Examinations with hemorrhage, ruptured aneurysm, posttraumatic or infectious pseudoaneurysm, arteriovenous malformation, surgical clips, coils, catheters, or other surgical hardware were excluded. All other CTA examinations were considered controls. Main Outcomes and Measures: Sensitivity, specificity, accuracy, time, and interrater agreement were measured. Metrics for clinician performance with and without model augmentation were compared. Results: The data set contained 818 examinations from 662 unique patients with 328 CTA examinations (40.1%) containing at least 1 intracranial aneurysm and 490 examinations (59.9%) without intracranial aneurysms. The 8 clinicians reading the test set ranged in experience from 2 to 12 years. Augmenting clinicians with artificial intelligence-produced segmentation predictions resulted in clinicians achieving statistically significant improvements in sensitivity, accuracy, and interrater agreement when compared with no augmentation. The clinicians' mean sensitivity increased by 0.059 (95% CI, 0.028-0.091; adjusted P = .01), mean accuracy increased by 0.038 (95% CI, 0.014-0.062; adjusted P = .02), and mean interrater agreement (Fleiss κ) increased by 0.060, from 0.799 to 0.859 (adjusted P = .05). There was no statistically significant change in mean specificity (0.016; 95% CI, -0.010 to 0.041; adjusted P = .16) and time to diagnosis (5.71 seconds; 95% CI, 7.22-18.63 seconds; adjusted P = .19). Conclusions and Relevance: The deep learning model developed successfully detected clinically significant intracranial aneurysms on CTA. This suggests that integration of an artificial intelligence-assisted diagnostic model may augment clinician performance with dependable and accurate predictions and thereby optimize patient care.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deep learning models have demonstrated diagnostic accuracy comparable to that of human experts in narrow clinical tasks for several medical domains and imaging modalities.
“deep learning models have demonstrated diagnostic accuracy comparable to that of human experts in narrow clinical tasks for several medical domains and imaging modalities, including chest and extremity X-rays8–10, computed tomography (CT)11, magnetic resonance imaging (MRI)12, whole slide images (WSI)13,14, and dermatology images15.”
From Self-supervised learning for medical image classification: a systematic review and implementation guidelines (2023) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., & Gerig, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.015 · Cited by 8,868 · see citing works
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · see citing works
Probable Inference, the Law of Succession, and Statistical Inference
Wilson, E. B. (1927) · Journal of the American Statistical Association · doi:10.1080/01621459.1927.10502953 · Cited by 4,476 · see citing works
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354 · see citing works
The Natural Course of Unruptured Cerebral Aneurysms in a Japanese Cohort
The UCAS Japan Investigators (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1113260 · Cited by 1,542 · see citing works
Rajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449 · see citing works
Self-citationOverview of deep learning in medical imaging
Suzuki, K. (2017) · Radiological Physics and Technology · doi:10.1007/s12194-017-0406-5 · Cited by 1,102 · see citing works
Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study
Chilamkurthy, S., Ghosh, R., Tanamala, S., Biviji, M., Campeau, N. G., Venugopal, V. K., Mahajan, V., Rao, P., & Warier, P. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31645-3 · Cited by 1,020 · see citing works
Bien, N., Rajpurkar, P., Ball, R. L., Irvin, J., Park, A., Jones, E., Bereket, M., Patel, B. N., Yeom, K. W., Shpanskaya, K., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002699 · Cited by 734 · see citing works
Self-citationAutomated deep-neural-network surveillance of cranial images for acute neurologic events
Titano, J. J., Badgeley, M., Schefflein, J., Pain, M., Su, A., Cai, M., Swinburne, N., Zech, J., Kim, J., Bederson, J., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0147-y · Cited by 458 · see citing works
White, P., Teasdale, E. M., Wardlaw, J. M., & Easton, V. (2001) · Radiology · doi:10.1148/radiology.219.3.r01ma16739 · Cited by 329 · see citing works
Hybrid 3D/2D Convolutional Neural Network for Hemorrhage Evaluation on Head CT
Chang, P. D., Kuoy, E., Grinband, J., Weinberg, B. D., Thompson, M., Homo, R. L., Chen, J. W., Abcede, H. G., Shafie, M., Sugrue, L. P., et al. (2018) · American Journal of Neuroradiology · doi:10.3174/ajnr.a5742 · Cited by 279 · see citing works
Deep Learning for MR Angiography: Automated Detection of Cerebral Aneurysms
Ueda, D., Yamamoto, A., Nishimori, M., Shimono, T., Doishita, S., Shimazaki, A., Katayama, Y., Fukumoto, S., Choppin, A., Shimahara, Y., et al. (2018) · Radiology · doi:10.1148/radiol.2018180901 · Cited by 233 · see citing works
Villablanca, J. P., Jahan, R., Hooshi, P., et al. (2002) · PubMed · Cited by 229 · see citing works
Detection of Intracranial Aneurysms: Multi–Detector Row CT Angiography Compared with DSA
Jayaraman, M. V., Mayo-Smith, W. W., Tung, G. A., et al. (2004) · Radiology · doi:10.1148/radiol.2302021465 · Cited by 215 · see citing works
Clinical Prediction Models for Aneurysmal Subarachnoid Hemorrhage: A Systematic Review
Jaja, B. N. R., Cusimano, M. D., Etminan, N., Hänggi, D., Hasan, D. M., Ilodigwe, D., Lantigua, H., Roux, P. L., Lo, B., Louffat-Olivares, A., et al. (2012) · Neurocritical Care · doi:10.1007/s12028-012-9792-z · Cited by 189 · see citing works
Lubicz, B., Levivier, M., François, O., et al. (2007) · American Journal of Neuroradiology · doi:10.3174/ajnr.a0699 · Cited by 131 · see citing works
Bone-subtraction CT angiography for the evaluation of intracranial aneurysms.
Tomandl, B. F., Hammen, T., Klotz, E., et al. (2006) · PubMed · Cited by 92 · see citing works
Lin, N., Ho, A. L., Gross, B. A., et al. (2012) · Journal of neurosurgery · doi:10.3171/2012.7.jns111766 · Cited by 84 · see citing works
Imaging of cerebral aneurysms: a clinical perspective
Yoon, N. K., McNally, J. S., Taussky, P., et al. (2016) · Neurovascular Imaging · doi:10.1186/s40809-016-0016-3 · Cited by 65 · see citing works
Turan, N., Heider, R. A., Roy, A., et al. (2018) · World Neurosurgery · doi:10.1016/j.wneu.2018.01.054 · Cited by 50 · see citing works
Bharatha, A., Yeung, R., Durant, D., et al. (2010) · Journal of Computer Assisted Tomography · doi:10.1097/rct.0b013e3181d27393 · Cited by 28 · see citing works
Shi, W., Li, Y.-D., Li, M.-H., et al. (2010) · Neurology India · doi:10.4103/0028-3886.73743 · Cited by 26 · see citing works
Maldaner, N., Stienen, M. N., Bijlenga, P., et al. (2017) · World Neurosurgery · doi:10.1016/j.wneu.2017.04.131 · Cited by 26 · see citing works
Yoon, Y. W., Park, S., Lee, S., et al. (2007) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000246896.89156.70 · Cited by 10 · see citing works
Residual aneurysm after metal coils treatment detected by spectral CT.
Wang, Y., Gao, X., Ai-xun, L., et al. (2012) · PubMed · doi:10.3978/j.issn.2223-4292.2012.06.04 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- [13]He K, Zhang X, Ren S, Sun J Deep residual learning for image recognition. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; June 27, 2016; Las Vegas, NV.
- [14]Xie S, Girshick R, Dollár P, Tu Z, He K Aggregated residual transformations for deep neural networks. Paper presented at: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); July 25, 2017; Honolulu, HI.
- [16]Ronneberger O, Fischer P, Brox T U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention Basel, Switzerland: Springer International; 2015:234–241.
- [17]Carreira J, Zisserman A Quo vadis, action recognition? a new model and the kinetics dataset. Paper presented at: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); July 25, 2017; Honolulu, HI.
- [18]Chen L-C, Papandreou G, Schroff F, Adam H Rethinking atrous convolution for semantic image segmentation. https://arxiv.org/abs/1706.05587. Published June 17, 2017. Accessed May 7, 2019. · arXiv
- [19]Milletari F, Navab N, Ahmadi S-A V-net: Fully convolutional neural networks for volumetric medical image segmentation. Paper presented at: 2016 IEEE Fourth International Conference on 3D Vision (3DV); October 26-28, 2016; Stanford, CA.
- [21]Huang G, Sun Y, Liu Z, Sedra D, Weinberger KQ Deep networks with stochastic depth. European Conference on Computer Vision Basel, Switzerland: Springer International; 2016:646–661.
- [35]Jnawali K, Arbabshirani MR, Rao N, Patel AA Deep 3D convolution neural network for CT brain hemorrhage classification. Paper presented at: Medical Imaging 2018: Computer-Aided Diagnosis. February 27, 2018; Houston, TX. doi: 10.1117/12.2293725 · DOI
Cited by
Sorted by citations
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · see citing works
Self-citationAI in diagnostic imaging: Revolutionising accuracy and efficiency
Khalifa, M., & Albadawy, M. (2024) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2024.100146 · Cited by 517 · see citing works
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479 · see citing works
“…accuracy comparable to that of human experts in narrow clinical tasks for several medical domains and imaging modalities, including chest and extremity X-rays 8 – 10 , computed tomography (CT) 11 , magnetic resonance imaging (MRI) 12 , whole slide images (WSI) 13,14 , and dermatology images 15 .”
BackgroundSelf-citationThe current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443 · see citing works
Self-citationRole of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419 · see citing works
Toğaçar, M., Ergen, B., & Cömert, Z. (2020) · Medical Hypotheses · doi:10.1016/j.mehy.2019.109531 · Cited by 310 · see citing works
Benchmarking saliency methods for chest X-ray interpretation
Saporta, A., Gui, X., Agrawal, A., Pareek, A., Truong, S. Q. H., Nguyen, C. D. T., Ngo, V.-D., Seekins, J., Blankenberg, F. G., Ng, A. Y., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00536-x · Cited by 270 · see citing works
Self-citationRajpurkar, P., O’Connell, C., Schechter, A., Asnani, N., Li, J., Kiani, A., Ball, R. L., Mendelson, M., Maartens, G., van Hoving, D. J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00322-2 · Cited by 137 · see citing works
Self-citationChi, E. A., Chi, G., Tsui, C. T., Jiang, Y., Jarr, K., Kulkarni, C. V., Zhang, M., Long, J., Ng, A. Y., Rajpurkar, P., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.17391 · Cited by 64 · see citing works
Self-citationJoel, M. Z., Umrao, S., Chang, E., Choi, R., Yang, D. X., Duncan, J. S., Omuro, A., Herbst, R., Krumholz, H. M., & Aneja, S. (2022) · JCO Clinical Cancer Informatics · doi:10.1200/cci.21.00170 · Cited by 51 · see citing works
- Authors
- Park, A., Chute, C., Rajpurkar, P., Lou, J., Ball, R. L., Shpanskaya, K., Jabarkheel, R., Kim, L. H., McKenna, E., Tseng, J., Ni, J., Wishah, F., Wittber, F., Hong, D. S., Wilson, T. J., Halabi, S., Basu, S., Patel, B. N., Lungren, M. P., Ng, A. Y., & Yeom, K. W.
- Affiliations
- Stanford University (US)
- Stanford Medicine (US)
- Corresponding author
- Yeom, Kristen W.
- ORCID
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2019-06-07
- Volume
- 2
- Issue
- 6
- Pages
- e195600
- ISSN
- 2574-3805 (ISSN-L)
- PMC
- PMC6563570
- OpenAlex
- W2948864380
- Cited by
- 258
- References
- 28
- Citations by year
- 2026: 16 · 2025: 34 · 2024: 41 · 2023: 39 · 2022: 48 · 2021: 42 · 2020: 32 · 2019: 5 · 2013: 1
- FWCI
- 21.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JAMA Network Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (UL1TR001085)
- National Center for Advancing Translational Sciences
- National Institute on Minority Health and Health Disparities (U54MD010724)
- Topics
- Health Sciences › Medicine › Neurology › Intracranial Aneurysms: Treatment and Complications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Cerebrovascular and Carotid Artery Diseases
- Keywords
- intracranial aneurysm, CT angiography, aneurysm segmentation, convolutional neural networks, computer-aided diagnosis, clinician performance, diagnostic accuracy, interobserver agreement, artificial intelligence in radiology, automated image analysis, time to diagnosis, clinical workflow integration
- MeSH terms
- Neurologists, Deep Learning, Intracranial Aneurysm, Clinical Competence, Computer Simulation, Diagnosis, Computer-Assisted, Female, Humans, Male, Middle Aged, Neurologic Examination, Retrospective Studies, Cross-Over Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
