The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index, and 1 are not checked yet.
Ultrasound image analysis using deep neural networks for discriminating between benign and malignant ovarian tumors: comparison with expert subjective assessment
Open accessAbstract
OBJECTIVES: To develop and test the performance of computerized ultrasound image analysis using deep neural networks (DNNs) in discriminating between benign and malignant ovarian tumors and to compare its diagnostic accuracy with that of subjective assessment (SA) by an ultrasound expert. METHODS: We included 3077 (grayscale, n = 1927; power Doppler, n = 1150) ultrasound images from 758 women with ovarian tumors, who were classified prospectively by expert ultrasound examiners according to IOTA (International Ovarian Tumor Analysis) terms and definitions. Histological outcome from surgery (n = 634) or long-term (≥ 3 years) follow-up (n = 124) served as the gold standard. The dataset was split into a training set (n = 508; 314 benign and 194 malignant), a validation set (n = 100; 60 benign and 40 malignant) and a test set (n = 150; 75 benign and 75 malignant). We used transfer learning on three pre-trained DNNs: VGG16, ResNet50 and MobileNet. Each model was trained, and the outputs calibrated, using temperature scaling. An ensemble of the three models was then used to estimate the probability of malignancy based on all images from a given case. The DNN ensemble classified the tumors as benign or malignant (Ovry-Dx1 model); or as benign, inconclusive or malignant (Ovry-Dx2 model). The diagnostic performance of the DNN models, in terms of sensitivity and specificity, was compared to that of SA for classifying ovarian tumors in the test set. RESULTS: At a sensitivity of 96.0%, Ovry-Dx1 had a specificity similar to that of SA (86.7% vs 88.0%; P = 1.0). Ovry-Dx2 had a sensitivity of 97.1% and a specificity of 93.7%, when designating 12.7% of the lesions as inconclusive. By complimenting Ovry-Dx2 with SA in inconclusive cases, the overall sensitivity (96.0%) and specificity (89.3%) were not significantly different from using SA in all cases (P = 1.0). CONCLUSION: Ultrasound image analysis using DNNs can predict ovarian malignancy with a diagnostic accuracy comparable to that of human expert examiners, indicating that these models may have a role in the triage of women with an ovarian tumor. © 2020 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“there were also studies with smaller sample sizes that showed a level of performance reaching those of human experts [188,191,192].”
The cited source says what the claim attributes to it.
In the citing paper
“Frequent described limitations of studies on AI applications in US imaging were homogeneity of data due to a single examiner or single center study [188,192]”
The cited source says what the claim attributes to it.
In the citing paper
“clinicians using AI image analyzing algorithms must still take clinical aspects into account [188,189].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Interval Estimation for a Binomial Proportion
Brown, L. D., Cai, T., & Dasgupta, A. (2001) · Statistical Science · doi:10.1214/ss/1009213286 · Cited by 3,746
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Flagged first, then by citations
Research works printed but not linked: 1
- Nesterov Y., 1983, A method of solving a convex programming problem with convergence rate O(1/k2), Sov Math Dokl, 27, 372
Cited by
Sorted by citations
Xu, H.-L., Gong, T., Liu, F.-H., Chen, H.-Y., Xiao, Q., Hou, Y., Huang, Y., Sun, H., Shi, Y., Gao, S., et al. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101662 · Cited by 135
Towards Clinical Application of Artificial Intelligence in Ultrasound Imaging
Komatsu, M., Sakai, A., Dozen, A., Shozu, K., Yasutomi, S., Machino, H., Asada, K., Kaneko, S., & Hamamoto, R. (2021) · Biomedicines · doi:10.3390/biomedicines9070720 · Cited by 122
A Deep Learning Fusion Approach to Diagnosis the Polycystic Ovary Syndrome (PCOS)
Alamoudi, A., Khan, I. U., Aslam, N., Alqahtani, N., Alsaif, H. S., Dandan, O. S. A., Gadeeb, M. A., & Bahrani, R. A. (2023) · Applied Computational Intelligence and Soft Computing · doi:10.1155/2023/9686697 · Cited by 111
Deep Learning Prediction of Ovarian Malignancy at US Compared with O-RADS and Expert Assessment
Chen, H., Yang, B.-W., Qian, L., Meng, Y.-S., Bai, X.-H., Hong, X.-W., He, X., Jiang, M.-J., Yuan, F., Du, Q.-W., et al. (2022) · Radiology · doi:10.1148/radiol.211367 · Cited by 110
Shrestha, P., Poudyal, B., Yadollahi, S., Wright, D., Gregory, A., Warner, J., Korfiatis, P. D., Green, I. C., Rassier, S. L. C., Mariani, A., et al. (2022) · Gynecologic Oncology · doi:10.1016/j.ygyno.2022.07.024 · Cited by 100
Metadata
- Authors
- Christiansen, F., Epstein, E. L., Smedberg, E., Åkerlund, M., Smith, K., & Epstein, E.
- Affiliations
- KTH Royal Institute of Technology (SE)
- Karolinska Institutet (SE)
- Harvard University (US)
- Science for Life Laboratory (SE)
- Corresponding author
- Epstein, E.
- Venue
- Ultrasound in Obstetrics & Gynecology
- Publisher
- Wiley
- Published
- 2021-01
- Volume
- 57
- Issue
- 1
- Pages
- 155-163
- ISSN
- 0960-7692 (ISSN-L)
- 1469-0705
- PMC
- PMC7839489
- OpenAlex
- W3096898300
- Cited by
- 142
- References
- 34
- Citations by year
- 2026: 17 · 2025: 27 · 2024: 46 · 2023: 31 · 2022: 13 · 2021: 8
- FWCI
- 5.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Ultrasound in Obstetrics and Gynecology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Radiumhemmets Forskningsfonder (181115)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- ovarian neoplasms, benign versus malignant differentiation, computer-aided diagnosis, deep neural network, transfer learning, ensemble learning, temperature scaling, subjective evaluation, power Doppler, B-mode ultrasound, ovarian cancer, diagnostic accuracy, histological outcomes, expert comparison, VGG16, ResNet50, MobileNet
- MeSH terms
- Deep Learning, Adnexal Diseases, Diagnosis, Differential, Female, Humans, Image Processing, Computer-Assisted, Ovarian Neoplasms, Retrospective Studies, Sensitivity and Specificity, Reproducibility of Results, Ultrasonography, Doppler
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
