Cited by 37Medical Image Segmentation TechniquesDomain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

Distributed deep learning across multisite datasets for generalized CT hemorrhage segmentation

Open access
Remedios, Samuel W., Roy, Snehashis, Bermudez, Camilo, Patel, Mayur B., Butman, John A., Landman, Bennett A., Pham, Dzung L.·2020·Medical Physics

Abstract

Purpose: As deep neural networks achieve more success in the wide field of computer vision, greater emphasis is being placed on the generalizations of these models for production deployment. With sufficiently large training datasets, models can typically avoid overfitting their data; however, for medical imaging it is often difficult to obtain enough data from a single site. Sharing data between institutions is also frequently nonviable or prohibited due to security measures and research compliance constraints, enforced to guard protected health information (PHI) and patient anonymity. Methods: In this paper, we implement cyclic weight transfer with independent datasets from multiple geographically disparate sites without compromising PHI. We compare results between single‐site learning (SSL) and multisite learning (MSL) models on testing data drawn from each of the training sites as well as two other institutions. Results: The MSL model attains an average dice similarity coefficient (DSC) of 0.690 on the holdout institution datasets with a volume correlation of 0.914, respectively corresponding to a 7% and 5% statistically significant improvement over the average of both SSL models, which attained an average DSC of 0.646 and average correlation of 0.871. Conclusions: We show that a neural network can be efficiently trained on data from two physically remote sites without consolidating patient data to a single location. The resulting network improves model generalization and achieves higher average DSCs on external datasets than neural networks trained on data from a single source.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/mp.13880

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Appropriately investigating variety of data sources can provide a strong indication for the applicability and generalisability of the ML application in different environments.

    “Appropriately investigating such questions can provide a strong indication for the applicability and generalisability of the ML application in different environments220–223.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. [1]Andreasen NC, Automatic atlas-based volume estimation of human brain regions from MR images, Journal of computer assisted tomography 20, 98– 106 (1996). 8576490
  2. [2]Lancaster J, Rainey L, Summerlin J, Freitas C, Fox P, Evans A, Toga A, and Mazziotta J, Automated labeling of the human brain: A preliminary report on the development and evaluation of a forward-transform method, Human brain mapping 5, 238– 242 (1997). 20408222 PMC2860189
  3. [3]Besag J, On the statistical analysis of dirty pictures, Journal of the Royal Statistical Society. Series B (Methodological), 259– 302 (1986).
  4. [4]Cootes TF and Taylor CJ, Statistical models of appearance for medical image analysis and computer vision, in Medical Imaging 2001: Image Processing, volume 4322, pages 236– 249, International Society for Optics and Photonics, 2001.
  5. [5]Pham DL, Xu C, and Prince JL, Current methods in medical image segmentation, Annual review of biomedical engineering 2, 315– 337 (2000). 11701515
  6. [6]Yahiaoui AFZ and Bessaid A, Segmentation of ischemic stroke area from CT brain images, in 2016 International Symposium on Signal, Image, Video and Communications (ISIVC), pages 13– 17, IEEE, 2016.
  7. [9]Andersson M and Knutsson H, Adaptive Spatio-temporal Filtering of 4D CT-Heart, in Scandinavian Conference on Image Analysis, pages 246– 255, Springer, 2013.
  8. [10]Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, and Sánchez CI, A survey on deep learning in medical image analysis, Medical image analysis 42, 60– 88 (2017). 28778026
  9. [11]Milletari F, Navab N, and Ahmadi S-A, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in 3D Vision (3DV), 2016 Fourth International Conference on, pages 565– 571, IEEE, 2016.
  10. [12]Prasoon A, Petersen K, Igel C, Lauze F, Dam E, and Nielsen M, Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network, in International conference on medical image computing and computer-assisted intervention, pages 246– 253, Springer, 2013. 24579147
  11. [14]Kooi T, Litjens G, van Ginneken B, Gubern-Merida A, Sánchez CI, Mann R, den Heeten A, and Karssemeijer N, Large scale deep learning for computer aided detection of mammographic lesions, Medical image analysis 35, 303– 312 (2017). 27497072
  12. [15]Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, and Langs G, Unsupervised anomaly detection with generative adversarial networks to guide marker discovery, in International Conference on Information Processing in Medical Imaging, pages 146– 157, Springer, 2017.
  13. [17]Szegedy C, Liu W, Jia Y, Sermanet P, Reed SE, Anguelov D, Erhan D, Vanhoucke V, and Rabinovich A, Going Deeper with Convolutions, pages 1– 9, 2015.
  14. [20]Chang P, Kuoy E, Grinband J, Weinberg B, Thompson M, Homo R, Chen J, Abcede H, Shafie M, Sugrue L, Filippi CG, Su M-Y, Yu W, Hess C, and D C, Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT, American Journal of Neuroradiology 39, 1609– 1616 (2018). 30049723 PMC6128745
  15. [21]Sudre CH, Anson BG, Ingala S, Lane CD, Jimenez D, Haider L, Varsavsky T, Smith L, Jager HR, and Cardoso MJ, 3D multirater RCNN for multimodal multiclass detection and characterisation of extremely small objects, arXiv preprint arXiv: 1812 09046 (2018).
  16. [22]Trifan G, Gattu R, Haacke EM, Kou Z, and Benson RR, MR imaging findings in mild traumatic brain injury with persistent neurological impairment, Magnetic resonance imaging 37, 243– 251 (2017). 27939436
  17. [23]Kinnunen KM, Greenwood R, Powell JH, Leech R, Hawkins PC, Bonnelle V, Patel MC, Counsell SJ, and Sharp DJ, White matter damage and cognitive impairment after traumatic brain injury, Brain 134, 449– 463 (2010). 21193486 PMC3030764
  18. [25]Halevy A, Norvig P, and Pereira F, The unreasonable effectiveness of data, IEEE Intelligent Systems 24, 8– 12 (2009).
  19. [26]Giles CL and Maxwell T, Learning, invariance, and generalization in high-order neural networks, Applied optics 26, 4972– 4978 (1987). 20523475
  20. [27]Yosinski J, Clune J, Bengio Y, and Lipson H, How transferable are features in deep neural networks?, in Advances in neural information processing systems, pages 3320– 3328, 2014.
  21. [28]Mackin D, Fave X, Zhang L, Fried D, Yang J, Taylor B, Rodriguez-Rivera E, Dodge C, Jones AK, and Court L, Measuring CT scanner variability of radiomics features, Investigative radiology 50, 757 (2015). 26115366 PMC4598251
  22. [29]Birnbaum BA, Hindman N, Lee J, and Babb JS, Multi-detector row CT attenuation measurements: assessment of intra-and interscanner variability with an anthropomorphic body CT phantom, Radiology 242, 109– 119 (2007). 17185663
  23. [30]A. Act, Health insurance portability and accountability act of 1996, Public law 104, 191 (1996). 16477734
  24. [32]Luxton DD, Kayl RA, and Mishkind MC, mHealth data security: The need for HIPAA-compliant standardization, Telemedicine and e-Health 18, 284– 288 (2012). 22400974
  25. [34]Fetzer DT and West OC, The HIPAA privacy rule and protected health information: implications in research involving DICOM image databases, Academic radiology 15, 390– 395 (2008). 18280936
  26. [35]Konecnỳ J, McMahan HB, Ramage D, and Richtárik P, Federated optimization: Distributed machine learning for on-device intelligence, arXiv preprint arXiv: 1610 02527 (2016).
  27. [36]Konečnỳ J, McMahan HB, Yu FX, Richtárik P, Suresh AT, and Bacon D, Federated learning: Strategies for improving communication efficiency, arXiv preprint arXiv: 1610 05492 (2016).
  28. [37]Zhang S, Zhang C, You Z, Zheng R, and Xu B, Asynchronous stochastic gradient descent for DNN training, in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pages 6660– 6663, IEEE, 2013.
  29. [39]Nguyen CV, Li Y, Bui TD, and Turner RE, Variational continual learning, arXiv preprint arXiv: 1710 10628 (2017).
  30. [40]Zenke F, Poole B, and Ganguli S, Continual learning through synaptic intelligence, in Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3987– 3995, JMLR. org, 2017. PMC6944509 31909397
  31. [41]McClure P, Zheng CY, Kaczmarzyk J, Rogers-Lee J, Ghosh S, Nielson D, Bandettini PA, and Pereira F, Distributed weight consolidation: A brain segmentation case study, in Advances in Neural Information Processing Systems, pages 4093– 4103, 2018. PMC8351531 34376963
  32. [42]Pan SJ, A survey on transfer learning, IEEE Transactions on knowledge and data engineering 22, 1345– 1359 (2010).
  33. [43]Chang K, Balachandar N, Lam C, Yi D, Brown J, Beers A, Rosen B, Rubin DL, and Kalpathy-Cramer J, Distributed deep learning networks among institutions for medical imaging, Journal of the American Medical Informatics Association (2018). PMC6077811 29617797
  34. [44]Remedios S, Roy S, Blaber J, Bermudez C, Nath V, Patel MB, Butman JA, Landman BA, and Pham DL, Distributed deep learning for robust multi-site segmentation of CT imaging after traumatic brain injury, in Medical Imaging 2018: Image Processing, volume 10949, 2019. PMC6786776 31602089
  35. [45]Muschelli J, Sweeney EM, Ullman NL, Vespa P, Hanley DF, and Crainiceanu CM, PItcHPERFeCT: primary intracranial hemorrhage probability estimation using random forests on CT, NeuroImage: Clinical 14, 379– 390 (2017). 28275541 PMC5328741
  36. [47]Gill J, Glial fibrillary acidic protein elevations relate to neuroimaging abnormalities after mild TBI, Neurology 91, e1385– e1389 (2018). 30209234 PMC6177279
  37. [48]Li X, Morgan PS, Ashburner J, Smith J, and Rorden C, The first step for neuroimaging data analysis: DICOM to NIfTI conversion, Journal of neuroscience methods 264, 47– 56 (2016). 26945974
  38. [49]Cox RW, AFNI: software for analysis and visualization of functional magnetic resonance neuroimages, Computers and Biomedical research 29, 162– 173 (1996). 8812068
  39. [50]Muschelli J, Ullman NL, Mould WA, Vespa P, Hanley DF, and Crainiceanu CM, Validated automatic brain extraction of head CT images, Neuroimage 114, 379– 385 (2015). 25862260 PMC4446187
  40. [51]Roy S, Butman JA, Chan L, and Pham DL, TBI contusion segmentation from MRI using convolutional neural networks, in Biomedical Imaging (ISBI 2018), 2018 IEEE 15th International Symposium on, pages 158– 162, IEEE, 2018.
  41. [52]Shamir RR, Duchin Y, Kim J, Sapiro G, and Harel N, Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentations, bioRxiv, 306977 (2018).
  42. [53]Kingma DP and Ba J, Adam: A method for stochastic optimization, arXiv preprint arXiv: 1412 6980 (2014).
  43. [54]Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, and Zhang L, Deep learning with differential privacy, in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308– 318, ACM, 2016.

Cited by

Sorted by citations

Authors
Remedios, S. W., Roy, S., Bermudez, C., Patel, M. B., Butman, J. A., Landman, B. A., & Pham, D. L.
Affiliations
  • Henry M. Jackson Foundation (US)
  • Vanderbilt University (US)
  • Critical Illness, Brain Dysfunction & Survivorship (CIBS) Center Section of Surgical Sciences Department of Surgery Division of Trauma Emergency General Surgery, & Surgical Critical Care Nashville TN 37212 USA
  • National Institutes of Health (US)
Corresponding author
Remedios, Samuel W.
Venue
Medical Physics
Publisher
Wiley
Published
2019-11-19
Volume
47
Issue
1
Pages
89-98
ISSN
  • 0094-2405 (ISSN-L)
  • 2473-4209
OpenAlex
W2982365645
Cited by
37
References
6
Citations by year
2026: 1 · 2025: 2 · 2024: 6 · 2023: 7 · 2022: 8 · 2021: 7 · 2020: 6
FWCI
2.35
Type
journal article
Language
en
Open access
Locations
  • Medical Physics (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Multiple Sclerosis Society (RG-1507-05243)
  • Patient-Centered Outcomes Research Institute (CDRN-1306-04869)
  • Nvidia
  • Vanderbilt University Medical Center (VR3029)
  • National Institute of General Medical Sciences (R01GM120484)
  • National Institute of Biomedical Imaging and Bioengineering (R01EB017230)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
distributed deep learning, model generalization, data privacy, protected health information, cross-institutional data sharing
MeSH terms
Deep Learning, Hemorrhage, Humans, Image Processing, Computer-Assisted, Tomography, X-Ray Computed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review