Automatic detection of oral cancer in smartphone-based images using deep learning for early diagnosis
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
SIGNIFICANCE: Oral cancer is a quite common global health issue. Early diagnosis of cancerous and potentially malignant disorders in the oral cavity would significantly increase the survival rate of oral cancer. Previously reported smartphone-based images detection methods for oral cancer mainly focus on demonstrating the effectiveness of their methodology, yet it still lacks systematic study on how to improve the diagnosis accuracy on oral disease using hand-held smartphone photographic images. AIM: We present an effective smartphone-based imaging diagnosis method, powered by a deep learning algorithm, to address the challenges of automatic detection of oral diseases. APPROACH: We conducted a retrospective study. First, a simple yet effective centered rule image-capturing approach was proposed for collecting oral cavity images. Then, based on this method, a medium-sized oral dataset with five categories of diseases was created, and a resampling method was presented to alleviate the effect of image variability from hand-held smartphone cameras. Finally, a recent deep learning network (HRNet) was introduced to evaluate the performance of our method for oral cancer detection. RESULTS: The performance of the proposed method achieved a sensitivity of 83.0%, specificity of 96.6%, precision of 84.3%, and F1 of 83.6% on 455 test images. The proposed "center positioning" method was about 8% higher than that of a simulated "random positioning" method in terms of F1 score, the resampling method had additional 6% of performance improvement, and the introduced HRNet achieved slightly better performance than VGG16, ResNet50, and DenseNet169, with respect to the metrics of sensitivity, specificity, precision, and F1. CONCLUSIONS: Capturing oral images centered on the lesion, resampling the cases in training set, and using the HRNet can effectively improve the performance of deep learning algorithm on oral cancer detection. The smartphone-based imaging with deep learning method has good potential for primary oral cancer diagnosis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We evaluated and compared a number of neural network architectures, including VGG16, ResNet50, DenseNet169, and HRNet-W18 on our collected dataset.”
ResultVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“We evaluated and compared a number of neural network architectures, including VGG16, ResNet50, DenseNet169, and HRNet-W18 on our collected dataset.”
ResultImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“39,40 We also applied common data augmentation, including left – right flipping, random resize cropping with ratio 0.7 to 1.0, arbitrary brightness, and contrast with ratio 0.9 to 1.1, and, for the model training.”
MethodInfluentialSung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., & Bray, F. (2021) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21660 · Cited by 108,699
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 4×
“To speed up the training and improve training performance, we used the preinitialization from ImageNet data-set.”
MethodInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“We utilized SGD 38 with a weight decay of 0.0001 and a Nesterov momentum of 0.9.”
MethodInfluentialDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“To compensate class imbalance, there are many solutions, such as using cost-modifying methods 27,28 to balance the learning and introducing GANs 29 to generate data scarcity samples.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“The initial learning rate was set to 0.001 and was reduced by 30% at epoch 15, 30, and 45.”
BackgroundInfluentialDeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · cited 1×
“The initial learning rate was set to 0.001 and was reduced by 30% at epoch 15, 30, and 45.”
BackgroundInfluentialAn introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 1×
“The corresponding AUCs were computed in each case using one versus other strategy.(44) The confidence intervals (CIs) of AUC at 95% were computed by the DeLong method.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
(2017) · Cited by 11,625
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Deep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2983686 · Cited by 4,894
Generative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477
The class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094 · cited 1×
“We observed that the over-sampling method decreases the model ’ s performance since it is susceptible to over-fitting of the minority classes 42,43 while our resampling method has an improvement of about 6% in F 1 score.”
MethodGenerative Adversarial Networks
(2022) · The Science of Deep Learning · doi:10.1017/9781108891530.013 · Cited by 2,402
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,945
Annual Report to the Nation on the Status of Cancer, 1975–2014, Featuring Survival
Jemal, A., Ward, E. M., Johnson, C. J., Cronin, K. A., Ma, J., Ryerson, A. B., Mariotto, A., Lake, A. J., Wilson, R., Sherman, R. L., et al. (2017) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djx030 · Cited by 1,556 · cited 1×
“The overall 5-year survival rate of diagnosed oral cancer patients is around 50% and has varied by race and area, 2 whereas survival rates as high as 65% have been reported in developed countries; in some rural areas, they can be as low as 15% depending on the affected part of the oral cavity.”
BackgroundOral Cancer and Precancerous Lesions
Neville, B. W., & Day, T. A. (2002) · CA: A Cancer Journal for Clinicians · doi:10.3322/canjclin.52.4.195 · Cited by 1,407 · cited 1×
“10,12 Diagnosing OPMDs as definable diseases is also challenging due to the numerous varieties, various forms, and overlapping features.”
BackgroundCancer Statistics, 2020: Report From National Cancer Registry Programme, India
Mathur, P., Sathishkumar, K., Chaturvedi, M., Das, P., Sudarshan, K. L., Santhappan, S., Nallasamy, V., John, A., Narasimhan, S., Roselind, F. S., et al. (2020) · JCO Global Oncology · doi:10.1200/go.20.00122 · Cited by 1,204 · cited 1×
“3 In short, patient survival rates and prognosis are severely compromised when oral cancer patients are diagnosed at more advanced stages 4,5 so that enhancing early diagnosis could mean a significant rise in positive survival outcomes.”
BackgroundPredicting cancer outcomes from histology and genomics using convolutional networks
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717139115 · Cited by 1,084 · cited 1×
“A variety of studies 17,18 showed that deep learning algorithms are able to surpass the performance of human experts in many disease recognition scenarios.”
BackgroundDeep, Big, Simple Neural Nets for Handwritten Digit Recognition
Cireşan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010) · Neural Computation · doi:10.1162/neco_a_00052 · Cited by 1,078 · cited 1×
“Data augmentation has been demonstrated to be an effective method for improving the performance of CNN with a limited training sample.(39,40) We also applied common data augmentation, including left– right flipping, random resize cropping with ratio 0.”
MethodHigh-Resolution Representations for Labeling Pixels and Regions
Sun, K., Zhao, Y. S., Jiang, B., Cheng, T., Xiao, B., Liu, D., Mu, Y., Wang, X., Liu, W., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04514 · Cited by 614 · cited 1×
“Recently, high-resolution representation learning has shown great potential in various vision applications, and related high-resolution networks (HRNet V1, HRNet V2, etc.) 31,32 have been proposed.”
BackgroundTschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R. P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., et al. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30333-x · Cited by 555
Molecular Pathogenesis of Oral Squamous Cell Carcinoma: Implications for Therapy
Choi, S., & Myers, J. N. (2008) · Journal of Dental Research · doi:10.1177/154405910808700104 · Cited by 516 · cited 1×
“6,7 As oral squamous cell carcinoma (OSCC) accounts for ∼ 90% of all oral cancer, 8,9 these two terms tend to be used interchangeably.”
BackgroundScreening programmes for the early detection and prevention of oral cancer
Brocklehurst, P., Kujan, O., O'Malley, L., Ogden, G. R., Shepherd, S., & Glenny, A.-M. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004150.pub4 · Cited by 459
Welikala, R. A., Remagnino, P., Lim, J. H., Chan, C. S., Rajendran, S., Kallarakkal, T. G., Zain, R. B., Jayasinghe, R. D., Rimal, J., Kerr, A. R., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3010180 · Cited by 372 · cited 3×
“The method allows direct focus on discriminative parts for disease recognition, without utilizing any region proposal methods, 24,25 or relying on bounding box annotation 24 by oral specialists.”
MethodInfluentialDeep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y. S., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.07919 · Cited by 326 · cited 1×
“Recently, high-resolution representation learning has shown great potential in various vision applications, and related high-resolution networks (HRNet V1, HRNet V2, etc.) 31,32 have been proposed.”
BackgroundAubreville, M., Knipfer, C., Oetter, N., Jaremenko, C., Rodner, E., Denzler, J., Bohr, C., Neumann, H., Stelzle, F., & Maier, A. (2017) · Scientific Reports · doi:10.1038/s41598-017-12320-8 · Cited by 326 · cited 1×
“Aubreville et al. 20 tested the deep convolutional neural network for OSCC diagnosis on CLE images, and the results showed that it outperformed the feature-based classification methods.”
MethodBoosted Classification Trees and Class Probability/Quantile Estimation
Mease, D., Wyner, A. J., & Buja, A. (2007) · ScholarlyCommons (University of Pennsylvania) · doi:10.5555/1248659.1248675 · Cited by 243
Uthoff, R. D., Song, B., Sunny, S., Patrick, S., Suresh, A., Kolur, T., Keerthi, G., Spires, O., Anbarani, A., Wilder-Smith, P., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0207493 · Cited by 192 · cited 2×
“Song et al. 21,22 developed mobile connected devices to acquire fluorescence oral images and used them to identify oral disease.”
BackgroundSong, B., Sunny, S., Uthoff, R. D., Patrick, S., Suresh, A., Kolur, T., Keerthi, G., Anbarani, A., Wilder-Smith, P., Kuriakose, M. A., et al. (2018) · Biomedical Optics Express · doi:10.1364/boe.9.005318 · Cited by 162 · cited 1×
“Song et al. 21,22 developed mobile connected devices to acquire fluorescence oral images and used them to identify oral disease.”
Çamalan, S., Mahmood, H., Binol, H., Araújo, A. L. D., Santos‐Silva, A. R., Vargas, P. A., Lopes, M. A., Khurram, S. A., & Gürcan, M. N. (2021) · Cancers · doi:10.3390/cancers13061291 · Cited by 142 · cited 2×
“Camalan et al. 23 used a CNN-based network for classifying white light images as normal or suspicious; however, the patient sample is very limited: with only 54 cases.”
BackgroundMethodDas, D. K., Bose, S., Maiti, A. K., Mitra, B., Mukherjee, G., & Dutta, P. K. (2018) · Tissue and Cell · doi:10.1016/j.tice.2018.06.004 · Cited by 120 · cited 1×
“For example, Kumar et al. 19 proposed a two-stage method that used a segmentation network and a random forest tree classifier to identify different stages of oral cancer in histological images.”
MethodScreening programmes for the early detection and prevention of oral cancer
Kujan, O., Glenny, A., Oliver, R., Thakker, N., & Sloan, P. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004150.pub2 · Cited by 89
Fine-Tuning Deep Learning Architectures for Early Detection of Oral Cancer
Welikala, R. A., Remagnino, P., Lim, J. H., Chan, C. S., Rajendran, S., Kallarakkal, T. G., Zain, R. B., Jayasinghe, R. D., Rimal, J., Kerr, A. R., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64511-3_3 · Cited by 24 · cited 2×
“The method allows direct focus on discriminative parts for disease recognition, without utilizing any region proposal methods, 24,25 or relying on bounding box annotation 24 by oral specialists.”
BackgroundMethod
Metadata
- Authors
- Lin, H., Chen, H., Weng, L., Shao, J., & Lin, J.
- Venue
- Journal of Biomedical Optics
- Publisher
- SPIE-Intl Soc Optical Eng
- Published
- 2021-08-28
- Volume
- 26
- Issue
- 08
- Pages
- 1-16
- ISSN
- 1083-3668 (ISSN-L)
- OpenAlex
- W3196077426
- Cited by
- 180
- References
- 39
- FWCI
- 8.38
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.