Automatic skin lesion segmentation using deep fully convolutional networks with jaccard distance.
Abstract
Automatic skin lesion segmentation in dermoscopic images is a challenging task due to the low contrast between lesion and the surrounding skin, the irregular and fuzzy lesion borders, the existence of various artifacts, and various imaging acquisition conditions. In this paper, we present a fully automatic method for skin lesion segmentation by leveraging 19-layer deep convolutional neural networks that is trained end-to-end and does not rely on prior knowledge of the data. We propose a set of strategies to ensure effective and efficient learning with limited training data. Furthermore, we design a novel loss function based on Jaccard distance to eliminate the need of sample re-weighting, a typical procedure when using cross entropy as the loss function for image segmentation due to the strong imbalance between the number of foreground and background pixels. We evaluated the effectiveness, efficiency, as well as the generalization capability of the proposed framework on two publicly available databases. One is from ISBI 2016 skin lesion analysis towards melanoma detection challenge, and the other is the PH2 database. Experimental results showed that the proposed method outperformed other state-of-the-art algorithms on these two databases. Our method is general enough and only needs minimum pre- and post-processing, which allows its adoption in a variety of medical image segmentation tasks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several algorithms incorporating artificial intelligence have been developed to help interpret clinical and dermoscopic images for a variety of skin conditions.
“Several algorithms incorporating artificial intelligence (AI) have been developed to help interpret both clinical11,12,13,14,15 and dermoscopic16,17,18,19,20,21,22,23 images for a variety of skin conditions”
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
(2014) · Cited by 33,419 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Siegel, R. L., Miller, K. D., & Jemal, A. (2016) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21332 · Cited by 16,003 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
(2010) · Cited by 14,134 · see citing works
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759 · see citing works
Final Version of 2009 AJCC Melanoma Staging and Classification
Balch, C. M., Gershenwald, J. E., Soong, S.-J., Thompson, J. F., Atkins, M. B., Byrd, D. R., Buzaid, A. C., Cochran, A. J., Coit, D. G., Ding, S., et al. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.23.4799 · Cited by 4,592 · see citing works
THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE. <sup>1</sup>
Jaccard, P. (1912) · New Phytologist · doi:10.1111/j.1469-8137.1912.tb05611.x · Cited by 4,561 · see citing works
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422 · see citing works
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · see citing works
(2013) · Cited by 3,740 · see citing works
Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images
Pereira, S., Pinto, A., Alves, V., & Silva, C. A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2538465 · Cited by 2,735 · see citing works
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., et al. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363547 · Cited by 2,620 · see citing works
Stochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_25 · Cited by 2,097 · see citing works
Static Analysis of Shape in TensorFlow ProgramsPreprint
The Theano Development Team, Al‐Rfou, R., Alain, G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2020.15 · Cited by 1,629 · see citing works
Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks
Cireşan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_51 · Cited by 1,536 · see citing works
Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks
Yu, L., Chen, H., Dou, Q., Qin, J., & Heng, P.-A. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2642839 · Cited by 1,140 · see citing works
PH<sup>2</sup> - A dermoscopic image database for research and benchmarking
Mendonca, T., Ferreira, P. M., Marques, J. S., Marcal, A. R. S., & Rozeira, J. (2013) · 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6610779 · Cited by 1,009 · see citing works
Vestergaard, M., Macaskill, P., Holt, P. E., & Menzies, S. W. (2008) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2008.08713.x · Cited by 896 · see citing works
A methodological approach to the classification of dermoscopy images
Celebi, M. E., Kingravi, H. A., Uddin, B., Iyatomi, H., Aslandogan, Y. A., Stoecker, W. V., & Moss, R. H. (2007) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2007.01.003 · Cited by 697 · see citing works
Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks
Dou, Q., Chen, H., Yu, L., Zhao, L., Qin, J., Wang, D., Mok, V. C., Shi, L., & Heng, P.-A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528129 · Cited by 679 · see citing works
Deep learning ensembles for melanoma recognition in dermoscopy images
Codella, N., Nguyen, Q.-B., Pankanti, S., Gutman, D. A., Helba, B., Halpern, A. C., & Smith, J. R. (2017) · IBM Journal of Research and Development · doi:10.1147/jrd.2017.2708299 · Cited by 665 · see citing works
Automated melanoma recognition
Ganster, H., Pinz, P., Röhrer, R., Wildling, E., Binder, M., & Kittler, H. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/42.918473 · Cited by 633 · see citing works
Avendi, M. R., Kheradvar, A., & Jafarkhani, H. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.01.005 · Cited by 591 · see citing works
A CNN Regression Approach for Real-Time 2D/3D Registration
Miao, S., Wang, Z. J., & Liao, R. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2521800 · Cited by 514 · see citing works
Lesion border detection in dermoscopy images
Celebi, M. E., Iyatomi, H., Schaefer, G., & Stoecker, W. V. (2009) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2008.11.002 · Cited by 423 · see citing works
Comparison of Segmentation Methods for Melanoma Diagnosis in Dermoscopy Images
Silveira, M., Nascimento, J. C., Marques, J. S., Marcal, A. R. S., Mendonça, T., Yamauchi, S., Maeda, J., & Rozeira, J. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2008.2011119 · Cited by 409 · see citing works
Gutman, D. A., Codella, N., Celebi, E., Helba, B., Marchetti, M. A., Mishra, N. K., & Halpern, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01397 · Cited by 365 · see citing works
Computerized analysis of pigmented skin lesions: A review
Korotkov, K., & García, R. (2012) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2012.08.002 · Cited by 355 · see citing works
Cha, K., Hadjiiski, L., Samala, R. K., Chan, H., Caoili, E., & Cohan, R. (2016) · Medical Physics (Lancaster) · doi:10.1118/1.4944498 · Cited by 248 · see citing works
Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition
Yan, Z., Zhan, Y., Peng, Z., Liao, S., Shinagawa, Y., Zhang, S., Metaxas, D., & Zhou, X. S. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2524985 · Cited by 223 · see citing works
Lesion Border Detection in Dermoscopy Images Using Ensembles of Thresholding Methods
Celebi, M. E., Wen, Q., Hwang, S., et al. (2012) · Skin Research and Technology · doi:10.1111/j.1600-0846.2012.00636.x · Cited by 207 · see citing works
A Novel Approach to Segment Skin Lesions in Dermoscopic Images Based on a Deformable Model
Ma, Z., & Tavares, J. M. R. S. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2390032 · Cited by 189 · see citing works
Combinations of Radiation Therapy and Immunotherapy for Melanoma: A Review of Clinical Outcomes
Barker, C. A., & Postow, M. A. (2014) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2013.08.035 · Cited by 161 · see citing works
Saliency-Based Lesion Segmentation Via Background Detection in Dermoscopic Images
Ahn, E., Kim, J., Bi, L., Kumar, A., Li, C., FULHAM, M. J., & Feng, D. D. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2653179 · Cited by 160 · see citing works
Xie, F., & Bovik, A. C. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.08.012 · Cited by 126 · see citing works
Mean shift based gradient vector flow for image segmentation
Zhou, H., Li, X., Schaefer, G., Celebi, M. E., & Miller, P. C. (2013) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2012.11.015 · Cited by 119 · see citing works
Segmentation of Dermoscopy Images Using Wavelet Networks
Sadri, A. R., Zekri, M., Sadri, S., Gheissari, N., Mokhtari, M., & Kolahdouzan, F. (2012) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2012.2227478 · Cited by 87 · see citing works
Simpler, Faster, More Accurate Melanocytic Lesion Segmentation Through MEDS
Peruch, F., Bogo, F., Bonazza, M., Cappelleri, V.-M., & Peserico, E. (2013) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2013.2283803 · Cited by 83 · see citing works
Bi, L., Kim, J., Ahn, E., Feng, D. D., & FULHAM, M. J. (2016) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493448 · Cited by 83 · see citing works
A State-of-the-Art Survey on Lesion Border Detection in Dermoscopy Images
Celebi, M. E., Mendonca, T., & Marques, J. S. (2015) · CRC Press eBooks · doi:10.1201/b19107-8 · Cited by 76 · see citing works
Automated saliency-based lesion segmentation in dermoscopic images
Ahn, E., Bi, L., Jung, Y. H., Kim, J., Li, C., FULHAM, M. J., & Feng, D. D. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7319025 · Cited by 62 · see citing works
Extraction of Skin Lesions from Non Dermoscopic Images Using Deep Learning
Begum, S., & Asra, S. (2017) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit1724124 · Cited by 16 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Image segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796 · see citing works
Survey on deep learning for radiotherapy
Meyer, P., Noblet, V., Mazzara, C., & Lallement, A. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.05.018 · Cited by 302 · see citing works
Federated learning improves site performance in multicenter deep learning without data sharing
Sarma, K. V., Harmon, S., Sanford, T., Roth, H. R., Xu, Z., Tetreault, J., Xu, D., Flores, M. G., Raman, A. G., Kulkarni, R., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa341 · Cited by 242 · see citing works
Skin disease diagnosis with deep learning: a review
Li, H., Pan, Y., Zhao, J., & Zhang, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.096 · Cited by 178 · see citing works
Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4
Zhou, J., He, X., Sun, L., Xu, J., Chen, X., Chu, Y., Zhou, L., Liao, X., Zhang, B., Afvari, S., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-50043-3 · Cited by 189 · see citing works
Jain, A., Way, D., Gupta, V., Gao, Y., de Oliveira Marinho, G., Hartford, J., Sayres, R., Kanada, K., Eng, C., Nagpal, K., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.7249 · Cited by 164 · see citing works
Melanoma Lesion Detection and Segmentation Using YOLOv4-DarkNet and Active Contour
Albahli, S., Nida, N., Irtaza, A., Yousaf, M. H., & Mahmood, M. T. (2020) · IEEE Access · doi:10.1109/access.2020.3035345 · Cited by 126 · see citing works
<i>auto</i>SMIM: Automatic Superpixel-Based Masked Image Modeling for Skin Lesion Segmentation
Wang, Z., Lyu, J., & Tang, X. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3290700 · Cited by 35 · see citing works
Mehta, D., Betz-Stablein, B., Nguyen, T. D., Gal, Y., Bowling, A., Haskett, M., Sashindranath, M., Bonnington, P., Mar, V., Soyer, H. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.01009 · Cited by 3 · see citing works
- Authors
- Yuan, Y., Chao, M., & Lo, Y.-C.
- Affiliations
- Icahn School of Medicine at Mount Sinai (US)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2017-09
- Volume
- 36
- Issue
- 9
- Pages
- 1876-1886
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2607363228
- Cited by
- 687
- References
- 49
- Citations by year
- 2026: 32 · 2025: 46 · 2024: 77 · 2023: 88 · 2022: 107 · 2021: 108 · 2020: 89 · 2019: 81 · 2018: 48 · 2017: 9
- FWCI
- 27.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- skin lesion segmentation, dermoscopic images, fully convolutional network, class imbalance, end-to-end learning, limited training data, melanoma detection, medical image segmentation
- MeSH terms
- Algorithms, Humans, Melanoma, Skin, Artifacts, Neural Networks, Computer, Dermoscopy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review