Cited by 147AI in cancer detectionRadiomics and Machine Learning in Medical ImagingArtificial Intelligence in Healthcare and Education

Ultrasound image analysis using deep neural networks for discriminating between benign and malignant ovarian tumors: Comparison with expert subjective assessment

Open access
Christiansen, F., Epstein, E. L., Smedberg, E., Åkerlund, M., Smith, K., Epstein, E.·2021·Ultrasound in Obstetrics & Gynecology

Abstract

OBJECTIVES: To develop and test the performance of computerized ultrasound image analysis using deep neural networks (DNNs) in discriminating between benign and malignant ovarian tumors and to compare its diagnostic accuracy with that of subjective assessment (SA) by an ultrasound expert. METHODS: We included 3077 (grayscale, n = 1927; power Doppler, n = 1150) ultrasound images from 758 women with ovarian tumors, who were classified prospectively by expert ultrasound examiners according to IOTA (International Ovarian Tumor Analysis) terms and definitions. Histological outcome from surgery (n = 634) or long-term (≥ 3 years) follow-up (n = 124) served as the gold standard. The dataset was split into a training set (n = 508; 314 benign and 194 malignant), a validation set (n = 100; 60 benign and 40 malignant) and a test set (n = 150; 75 benign and 75 malignant). We used transfer learning on three pre-trained DNNs: VGG16, ResNet50 and MobileNet. Each model was trained, and the outputs calibrated, using temperature scaling. An ensemble of the three models was then used to estimate the probability of malignancy based on all images from a given case. The DNN ensemble classified the tumors as benign or malignant (Ovry-Dx1 model); or as benign, inconclusive or malignant (Ovry-Dx2 model). The diagnostic performance of the DNN models, in terms of sensitivity and specificity, was compared to that of SA for classifying ovarian tumors in the test set. RESULTS: At a sensitivity of 96.0%, Ovry-Dx1 had a specificity similar to that of SA (86.7% vs 88.0%; P = 1.0). Ovry-Dx2 had a sensitivity of 97.1% and a specificity of 93.7%, when designating 12.7% of the lesions as inconclusive. By complimenting Ovry-Dx2 with SA in inconclusive cases, the overall sensitivity (96.0%) and specificity (89.3%) were not significantly different from using SA in all cases (P = 1.0). CONCLUSION: Ultrasound image analysis using DNNs can predict ovarian malignancy with a diagnostic accuracy comparable to that of human expert examiners, indicating that these models may have a role in the triage of women with an ovarian tumor. © 2020 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/uog.23530

Cites

34 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [26]Nesterov Y. A method of solving a convex programming problem with convergence rate O(1/k2). Sov Math Dokl 1983; 27: 372– 376.
  2. [27]Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France, 2015; 37: 448– 456. https://arxiv.org/abs/1502.03167. · arXiv
  3. [32]Ojala T, Pietikäinen M, Harwood D. Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. Proceedings of 12th International Conference on Pattern Recognition (ICPR), Jerusalem, Israel, 1994; 1: 582– 585.

Cited by

Sorted by citations

Authors
Christiansen, F., Epstein, E. L., Smedberg, E., Åkerlund, M., Smith, K., & Epstein, E.
Affiliations
  • KTH Royal Institute of Technology (SE)
  • Karolinska Institutet (SE)
  • Harvard University (US)
  • Science for Life Laboratory (SE)
Corresponding author
Epstein, E.
Venue
Ultrasound in Obstetrics & Gynecology
Publisher
Wiley
Published
2021-01
Volume
57
Issue
1
Pages
155-163
ISSN
  • 0960-7692 (ISSN-L)
  • 1469-0705
OpenAlex
W3096898300
Cited by
142
References
34
Citations by year
2026: 17 · 2025: 27 · 2024: 46 · 2023: 31 · 2022: 13 · 2021: 8
FWCI
5.11
Type
journal article
Language
en
Open access
Locations
  • Ultrasound in Obstetrics and Gynecology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Radiumhemmets Forskningsfonder (181115)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ovarian neoplasms, benign versus malignant differentiation, computer-aided diagnosis, deep neural network, transfer learning, ensemble learning, temperature scaling, subjective evaluation, power Doppler, B-mode ultrasound, ovarian cancer, diagnostic accuracy, histological outcomes, expert comparison, VGG16, ResNet50, MobileNet
MeSH terms
Deep Learning, Adnexal Diseases, Diagnosis, Differential, Female, Humans, Image Processing, Computer-Assisted, Ovarian Neoplasms, Retrospective Studies, Sensitivity and Specificity, Reproducibility of Results, Ultrasonography, Doppler

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review