The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Deep Learning-Based Image Segmentation on Multimodal Medical Imaging
Open accessAbstract
Multi-modality medical imaging techniques have been increasingly applied in clinical practice and research studies. Corresponding multi-modal image analysis and ensemble learning schemes have seen rapid growth and bring unique value to medical applications. Motivated by the recent success of applying deep learning methods to medical image processing, we first propose an algorithmic architecture for supervised multi-modal image analysis with cross-modality fusion at the feature learning level, classifier level, and decision-making level. We then design and implement an image segmentation system based on deep Convolutional Neural Networks (CNN) to contour the lesions of soft tissue sarcomas using multi-modal images, including those from Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and Positron Emission Tomography (PET). The network trained with multi-modal images shows superior performance compared to networks trained with single-modal images. For the task of tumor segmentation, performing image fusion within the network (i.e. fusing at convolutional or fully connected layers) is generally better than fusing images at the network output (i.e. voting). This study provides empirical guidance for the design and application of multi-modal image analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Measures of the Amount of Ecologic Association Between Species
Dice, L. R. (1945) · Ecology · doi:10.2307/1932409 · Cited by 11,874
The random subspace method for constructing decision forests
Ho, T. K. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.709601 · Cited by 6,842
The multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781
Deep learning in medical image analysis
Shen, D., Wu, G., & Suk, H.-I. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071516-044442 · Cited by 4,968
The Strength of Weak Learnability
Schapire, R. E. (1990) · Machine Learning · doi:10.1007/bf00116037 · Cited by 4,600
The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository
Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Maffitt, D., Pringle, M., et al. (2013) · Journal of Digital Imaging · doi:10.1007/s10278-013-9622-7 · Cited by 4,599
Brain tumor segmentation with Deep Neural Networks
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., & Larochelle, H. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.05.004 · Cited by 3,278
(2011) · Cited by 2,752
Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images
Pereira, S., Pinto, A., Alves, V., & Silva, C. A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2538465 · Cited by 2,735
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657
A combined PET/CT scanner for clinical oncology.
Beyer, T., Townsend, D. W., Brun, T., Kinahan, P. E., Charron, M., Roddy, R., Jerin, J., Young, J., Byars, L. G., & Nutt, R. (2000) · PubMed · Cited by 1,659
Casali, P. G., Abecassis, N., Bauer, S., Biagini, R., Bielack, S., Bonvalot, S., Boukovinas, I., Bovee, J. V. M. G., Brodowicz, T., Broto, J. M., et al. (2018) · Annals of Oncology · doi:10.1093/annonc/mdy096 · Cited by 1,614
Deformable medical image registration: a survey
Sotiras, A., Davatzikos, C., & Paragios, N. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2265603 · Cited by 1,587 · cited 1×
“Yet we also note that there exist unsupervised methods in medical image analysis such as gradient flow-based methods for image segmentation [47], as well as well-established deformable image registration algorithms [48].”
MethodHierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis
Suk, H.-I., Lee, S.-W., & Shen, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.077 · Cited by 968 · cited 1×
“The multimodal feature representation framework introduced in [26] fuses information from MRI and PET in a hierarchical deep learning approach.”
BackgroundMedical image fusion: A survey of the state of the art
James, A. P., & Dasarathy, B. V. (2014) · Information Fusion · doi:10.1016/j.inffus.2013.12.002 · Cited by 967
Vallières, M., Freeman, C. R., Skamene, S. R., & El Naqa, I. (2015) · Physics in Medicine and Biology · doi:10.1088/0031-9155/60/14/5471 · Cited by 897
On Deep Multi-View Representation Learning
Wang, W., Livescu, K., & Bilmes, J. (2015) · Cited by 752
Thrall, J. H., Li, X., Li, Q., Cruz, C., Do, S., Dreyer, K., & Brink, J. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.12.026 · Cited by 710
Self-citation(2014) · Annals of Oncology · doi:10.1093/annonc/mdu254 · Cited by 656
Christ, P. F., Elshaer, M. E. A., Ettlinger, F., Tatavarty, S., Bickel, M., Bilic, P., Rempfler, M., Armbruster, M., Hofmann, F., D’Anastasi, M., et al. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_48 · Cited by 625
Wang, S., Zhou, M., Liu, Z., Liu, Z., Gu, D., Zang, Y., Dong, D., Gevaert, O., & Tian, J. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.06.014 · Cited by 478
Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain
Bhatnagar, G., Wu, Q. M. J., & Liu, Z. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2244870 · Cited by 471
Guidelines for the Management of Soft Tissue Sarcomas
Grimer, R., Judson, I., Peake, D., & Seddon, B. (2010) · Sarcoma · doi:10.1155/2010/506182 · Cited by 432
Klein, S., van der Heide, U. A., Lips, I. M., van Vulpen, M., Staring, M., & Pluim, J. P. W. (2008) · Medical Physics · doi:10.1118/1.2842076 · Cited by 414
Benign and malignant breast tumors classification based on region growing and CNN segmentation
Rouhi, R., Shahbazzadeh, M. J., Shohreh Kasaei, B., & Keshavarzian, P. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.09.020 · Cited by 394
Singh, R., & Khare, A. (2014) · Information Fusion · doi:10.1016/j.inffus.2012.09.005 · Cited by 332
Unregistered Multiview Mammogram Analysis with Pre-trained Deep Learning Models
Carneiro, G., Nascimento, J., & Bradley, A. P. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24574-4_78 · Cited by 235
Integrative data analysis of multi-platform cancer data with a multimodal deep learning approach
Liang, M., Li, Z., Chen, T., & Zeng, J. (2015) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2014.2377729 · Cited by 231
Teramoto, A., Fujita, H., Yamamuro, O., & Tamaki, T. (2016) · Medical Physics · doi:10.1118/1.4948498 · Cited by 226
Zhu, X., Suk, H.-I., Lee, S.-W., & Shen, D. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2466616 · Cited by 218
Czisch, M., Wetter, T. C., Kaufmann, C., Pollmächer, T., Holsboer, F., & Auer, D. P. (2002) · NeuroImage · doi:10.1006/nimg.2002.1071 · Cited by 207
Multimodal Deep Learning for Cervical Dysplasia Diagnosis
Xu, T., Zhang, H., Huang, X., Zhang, S., & Metaxas, D. N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_14 · Cited by 187
Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules
Feng, X., Yang, J., Laine, A. F., & Angelini, E. D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_65 · Cited by 168
Bagci, U., Udupa, J. K., Mendhiratta, N., Foster, B., Xu, Z., Yao, J., Chen, X., & Mollura, D. J. (2013) · Medical Image Analysis · doi:10.1016/j.media.2013.05.004 · Cited by 159
Correa, N. M., Eichele, T., Adalı, T., Li, Y.-O., & Calhoun, V. D. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.062 · Cited by 151
3D cell nuclei segmentation based on gradient flow tracking
Li, G., Liu, T., Tarokh, A. B., Nie, J., Guo, L., Mara, A., Holley, S. A., & Wong, S. T. C. (2007) · BMC Cell Biology · doi:10.1186/1471-2121-8-40 · Cited by 150
Multimodal medical image fusion using PCNN optimized by the QPSO algorithm
Xu, X., Shan, D., Wang, G., & Jiang, X. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2016.03.028 · Cited by 125
Technical Note: A deep learning‐based autosegmentation of rectal tumors in MR images
Wang, J., Lu, J., Qin, G., Shen, L., Sun, Y., Ying, H., Zhang, Z., & Hu, W. (2018) · Medical Physics · doi:10.1002/mp.12918 · Cited by 112
Deep Learning to Hash with Multiple Representations
Kang, Y., Kim, S., & Choi, S. (2012) · Proceedings · doi:10.1109/icdm.2012.24 · Cited by 67
Multi-source deep learning for information trustworthiness estimation
Ge, L., Gao, J., Xiao-yi, L., & Zhang, A. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487612 · Cited by 44
RBC Semantic Segmentation for Sickle Cell Disease Based on Deformable U-Net
Zhang, M., Li, X., Xu, M., & Li, Q. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_79 · Cited by 41
Self-citationPROBABILISTIC SEGMENTATION OF BRAIN TUMORS BASED ON MULTI-MODALITY MAGNETIC RESONANCE IMAGES
Cai, H., Verma, R., Ou, Y., Lee, S.-K., Melhem, E. R., & Davatzikos, C. A. (2007) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2007.356923 · Cited by 34
Lorenzi, M., Simpson, I., Mendelson, A., Vos, S. B., Cardoso, M. J., Marc Modat, A., Schott, J. M., & Sebastien Ourselin, A. (2016) · Scientific Reports · doi:10.1038/srep22161 · Cited by 24
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
A review: Deep learning for medical image segmentation using multi-modality fusion
Zhou, T., Ruan, S., & Canu, S. (2019) · Array · doi:10.1016/j.array.2019.100004 · Cited by 656
On Interpretability of Artificial Neural Networks: A Survey
Fan, F.-L., Xiong, J., Li, M., & Wang, G. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3066428 · Cited by 561
Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey
Bhattacharya, S., Reddy Maddikunta, P. K., Pham, Q.-V., Gadekallu, T. R., Krishnan S, S. R., Chowdhary, C. L., Alazab, M., & Jalil Piran, M. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2020.102589 · Cited by 483
Artificial intelligence and machine learning for medical imaging: A technology review
Barragán-Montero, A., Javaid, U., Valdés, G., Nguyen, D., Desbordes, P., Macq, B., Willems, S., Vandewinckele, L., Holmström, M., Löfman, F., et al. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.04.016 · Cited by 450
Metadata
- Authors
- Guo, Z., Li, X., Huang, H., Guo, N., & Li, Q.
- Venue
- IEEE Transactions on Radiation and Plasma Medical Sciences
- Published
- 2019-03
- Volume
- 3
- Issue
- 2
- Pages
- 162-169
- ISSN
- 2469-7303 (ISSN-L)
- 2469-7311
- OpenAlex
- W2907760128
- Cited by
- 440
- References
- 47
- FWCI
- 21.12
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.