Generalizable Multi-Site Training and Testing Of Deep Neural Networks Using Image Normalization
Open accessAbstract
The ability of medical image analysis deep learning algorithms to generalize across multiple sites is critical for clinical adoption of these methods. Medical imging data, especially MRI, can have highly variable intensity characteristics across different individuals, scanners, and sites. However, it is not practical to train algorithms with data from all imaging equipment sources at all possible sites. Intensity normalization methods offer a potential solution for working with multi-site data. We evaluate five different image normalization methods on training a deep neural network to segment the prostate gland in MRI. Using 600 MRI prostate gland segmentations from two different sites, our results show that both intra-site and inter-site evaluation is critical for assessing the robustness of trained models and that training with single-site data produces models that fail to fully generalize across testing data from sites not included in the training.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Appropriately investigating variety of data sources can provide a strong indication for the applicability and generalisability of the ML application in different environments.
“Appropriately investigating such questions can provide a strong indication for the applicability and generalisability of the ML application in different environments220–223.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [221]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study
Chilamkurthy, S., Ghosh, R., Tanamala, S., Biviji, M., Campeau, N. G., Venugopal, V. K., Mahajan, V., Rao, P., & Warier, P. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31645-3 · Cited by 1,020 · see citing works
Evaluation of prostate segmentation algorithms for mri: The promise12 challenge
Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., et al. (2014) · Medical Image Analysis · doi:10.1016/j.media.2013.12.002 · Cited by 857 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- [1]Litjens G, Kooi T, Bejnordi BE, Setio AAA,, “ A survey on deep learning in medical image analysis,” Med. Image Anal, vol. 42, pp. 60– 88, 2017. 28778026
- [2]Pan SJ and Yang Q, “ A Survey on Transfer Learning,” IEEE Trans. Knowl. Data Eng, vol. 22, no. 10, pp. 1345– 1359, 2010.
- [3]Li Z and Hoiem D, “ Learning without Forgetting,” IEEE Trans. Pattern Anal. Mach. Intell, 2018. 29990101
- [6]Cannon TD, “ Reliability of neuroanatomical measurements in a multisite longitudinal study of youth at risk for psychosis,” Hum Brain Mapp, vol. 35, no. 5, pp. 2424– 34, 2014. 23982962 PMC3843968
- [8]Gibson E, “ Inter-site variability in prostate segmentation accuracy using deep learning,” in MICCAI, pp. 506– 514. Springer, 2018.
- [9]Nyul LG and Udupa JK, “ On standardizing the MR image intensity scale,” Mag. Res. Med, vol. 42, no. 6, pp. 1072– 1081, 1999. 10571928
- [11]Ladak HM, Mao F, Wang Y, Downey DB, Steinman DA, and Fenster A, “ Prostate boundary segmentation from 2D ultrasound images,” Med. Phys, vol. 27, no. 8, pp. 1777– 1788, 2000. 10984224
Cited by
Sorted by citations
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169 · see citing works
Ben Ahmed, K., Hall, L. O., Goldgof, D. B., & Fogarty, R. (2022) · IEEE Access · doi:10.1109/access.2022.3193700 · Cited by 12 · see citing works
- Authors
- Onofrey, J. A., Casetti-Dinescu, D. I., Lauritzen, A. D., Sarkar, S., Venkataraman, R., Fan, R. E., Sonn, G. A., Sprenkle, P. C., Staib, L. H., & Papademetris, X.
- Affiliations
- Yale University (US)
- Stanford University (US)
- ORCID
- Venue
- 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)
- Publisher
- IEEE
- Published
- 2019-04
- Volume
- 2019
- Pages
- 348-351
- PMC
- PMC7457546
- OpenAlex
- W2959556310
- Cited by
- 52
- References
- 4
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 4 · 2023: 15 · 2022: 12 · 2021: 12 · 2020: 5 · 2019: 1
- FWCI
- 2.45
- Type
- conference paper
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- IEEE International Symposium on Biomedical Imaging (ISBI) (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Cancer Institute (R41CA224888, R42CA224888)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- image normalization, intensity normalization, prostate segmentation, magnetic resonance imaging, deep neural network, inter-site variability, scanner variability, model robustness, medical image analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
