Deep learning for segmentation of brain tumors: Impact of cross‐institutional training and testing
Abstract
Background and purpose Convolutional neural networks (CNNs) are commonly used for segmentation of brain tumors. In this work, we assess the effect of cross-institutional training on the performance of CNNs. Methods We selected 44 glioblastoma (GBM) patients from two institutions in The Cancer Imaging Archive dataset. The images were manually annotated by outlining each tumor component to form ground truth. To automatically segment the tumors in each patient, we trained three CNNs: (a) one using data for patients from the same institution as the test data, (b) one using data for the patients from the other institution and (c) one using data for the patients from both of the institutions. The performance of the trained models was evaluated using Dice similarity coefficients as well as Average Hausdorff Distance between the ground truth and automatic segmentations. The 10-fold cross-validation scheme was used to compare the performance of different approaches. Results Performance of the model significantly decreased (P Conclusions There is a very strong effect of selecting data for training on performance of CNNs in a multi-institutional setting. Determination of the reasons behind this effect requires additional comprehensive investigation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Variations in data collection and processing can degrade model accuracy on data from new hospital deployments.
“However, variations in data collection and processing can degrade model accuracy on data from new hospital deployments (Zech et al., 2018; AlBadawy et al., 2018).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as AlBadawy et al., 2018
Models for medical applications are often trained on data from a small number of hospitals but deployed more generally, and variations in data collection and processing can degrade model accuracy on data from new hospital deployments.
“Models for medical applications are often trained on data from a small number of hospitals, but with the goal of being deployed more generally across other hospitals. However, variations in data collection and processing can degrade model accuracy on data from new hospital deployments (Zech et al., 2018; AlBadawy et al., 2018).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as AlBadawy et al., 2018
Cites
Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · see citing works
Brain tumor segmentation with Deep Neural Networks
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., & Larochelle, H. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.05.004 · Cited by 3,278 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058 · see citing works
Domain adaptation for medical image analysis: A survey
Guan, H., & Liu, M. (2022) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3117407 · Cited by 1,057 · see citing works
Kim, D. W., Jang, H. Y., Kim, K. W., Shin, Y., & Park, S. H. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2019.0025 · Cited by 459 · see citing works
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443 · see citing works
Bai, H. X., Wang, R., Xiong, Z., Hsieh, B., Chang, K., Halsey, K., Tran, T. M. L., Choi, J. W., Wang, D.-C., Shi, L.-B., et al. (2020) · Radiology · doi:10.1148/radiol.2020201491 · Cited by 381 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
“The challenge is that variation in data collection and processing across different hospitals can degrade model accuracy on data from hospitals not included in the training set (e.g., Zech et al. (2018); AlBadawy et al. (2018)).”
Data drift in medical machine learning: implications and potential remedies
Sahiner, B., Chen, W., Samala, R. K., & Petrick, N. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20220878 · Cited by 279 · see citing works
Multi-institutional validation of a mammography-based breast cancer risk model
Yala, A., Mikhael, P. G., Strand, F., Lin, G., Satuluru, S., Kim, T., Banerjee, I., Gichoya, J., Trivedi, H., Lehman, C. D., et al. (2022) · Journal of Clinical Oncology · doi:10.1200/jco.21.01337 · Cited by 204 · see citing works
Mårtensson, G., Ferreira, D., Granberg, T., Cavallin, L., Oppedal, K., Padovani, A., Rektorova, I., Bonanni, L., Pardini, M., Kramberger, M. G., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101714 · Cited by 176 · see citing works
Advancing medical imaging informatics by deep learning-based domain adaptation
Choudhary, A., Tong, L., Zhu, Y., & Wang, M. D. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1702009 · Cited by 84 · see citing works
In Search of Lost Domain Generalization
Gulrajani, I., & Lopez-Paz, D. (2020) · arXiv · doi:10.48550/arxiv.2007.01434 · Cited by 440 · see citing works
“As another example, systems trained on medical data collected in one hospital do not generalize to other health centers [Castro et al., 2019, AlBadawy et al., 2018, Perone et al., 2019, Heaven, 2020].”
ResultPooch, E. H. P., Ballester, P. L., & Barros, R. C. (2019) · arXiv · doi:10.48550/arxiv.1909.01940 · Cited by 32 · see citing works
Sharafutdinov, K., Bhat, J. S., Fritsch, S. J., Nikulina, K., E. Samadi, M., Polzin, R., Mayer, H., Marx, G., Bickenbach, J., & Schuppert, A. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.603429 · Cited by 10 · see citing works
- Authors
- AlBadawy, E. A., Saha, A., & Mazurowski, M. A.
- Affiliations
- Duke University (US)
- Corresponding author
- Mazurowski, Maciej A.
- Venue
- Medical Physics
- Publisher
- Wiley
- Published
- 2018-02-08
- Volume
- 45
- Issue
- 3
- Pages
- 1150-1158
- ISSN
- 0094-2405 (ISSN-L)
- 2473-4209
- OpenAlex
- W2791655542
- Cited by
- 208
- References
- 2
- Citations by year
- 2026: 14 · 2025: 24 · 2024: 34 · 2023: 27 · 2022: 21 · 2021: 40 · 2020: 26 · 2019: 15 · 2018: 7
- FWCI
- 9.87
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Physics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Neurology › Brain Tumor Detection and Classification
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- brain tumor segmentation, glioblastoma, convolutional neural networks, multi-institutional data, training data selection, model generalization, The Cancer Imaging Archive
- MeSH terms
- Machine Learning, Brain Neoplasms, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
