PreprintCited by 315Artificial Intelligence in Healthcare and EducationCongenital Heart Disease StudiesArtificial Intelligence in Healthcare

Expert-level prenatal detection of complex congenital heart disease from screening ultrasound using deep learning

Open access
Arnaout, Rima, Curran, Lara, Zhao, Yili, Levine, Jami C., Chinn, Erin, Moon-Grady, Anita J.·2020·Nature Medicine

Abstract

Congenital heart disease (CHD) is the most common birth defect. Fetal survey ultrasound is recommended worldwide, including five views of the heart that together could detect 90% of complex CHD. In practice, however, sensitivity is as low as 30%. We hypothesized poor detection results from challenges in acquiring and interpreting diagnostic-quality cardiac views, and that deep learning could improve complex CHD detection. Using 107,823 images from 1,326 retrospective echocardiograms and surveys from 18-24 week fetuses, we trained an ensemble of neural networks to (i) identify recommended cardiac views and (ii) distinguish between normal hearts and complex CHD. Finally, (iii) we used segmentation models to calculate standard fetal cardiothoracic measurements. In a test set of 4,108 fetal surveys (0.9% CHD, >4.4 million images, about 400 times the size of the training dataset) the model achieved an AUC of 0.99, 95% sensitivity (95%CI, 84-99), 96% specificity (95%CI, 95-97), and 100% NPV in distinguishing normal from abnormal hearts. Sensitivity was comparable to clinicians’ task-for-task and remained robust on external and lower-quality images. The model’s decisions were based on clinically relevant features. Cardiac measurements correlated with reported measures for normal and abnormal hearts. Applied to guidelines-recommended imaging, ensemble learning models could significantly improve detection of fetal CHD and expand telehealth options for prenatal care at a time when the COVID-19 pandemic has further limited patient access to trained providers. This is the first use of deep learning to ∼double standard clinical performance on a critical and global diagnostic challenge.

Verification finished with gaps

Some checks didn’t finish. The results below cover the steps that did.

  1. Registry recordVerified
  2. Full textRead the full text
  3. Reference list51 references found
  4. Reference check53 of 53 references checked
  5. StatisticsRecomputed
  6. ClaimsDidn’t finish — the rest of the report still applies.
  7. Cited sourcesDidn’t finish — the rest of the report still applies.
  8. Self-checkChecked
  9. VerificationFindings verified

AI snapshot

Written from the paper’s full text.

TL;DR

An ensemble of deep learning models was trained on fetal echocardiograms and screening ultrasounds to identify recommended cardiac views, distinguish normal hearts from complex congenital heart disease (CHD), and derive cardiothoracic measurements. In a large internal screening-ultrasound test set with population-like CHD prevalence, the model achieved an AUC of 0.99 with 95% sensitivity and 96% specificity.

Key findings

  • Using up to 107,823 images from up to 1,326 studies, the ensemble achieved an AUC of 0.99 in OB-4000, a test set of 4,108 fetal surveys with 0.9% CHD and >4.4 million images; sensitivity was 95% (95% CI, 84–99%), specificity 96% (95% CI, 95–97%), positive predictive value 20% (95% CI, 17–23%), and negative predictive value 100%.
  • View classification on FETAL-125 had an F score of 0.93 (AUC range, 0.94–0.98); on OB-125, sensitivity was 90% and specificity 78%, rising to 96% and 92% when using images with prediction probabilities at or above the first quartile.
  • On external BCH-400 fetal echocardiograms with 92% CHD, composite classification AUC was 0.89; on TWINS-10, sensitivity was 100% and specificity 72%; for specific lesions, sensitivity was 71% for tetralogy of Fallot and 89% for hypoplastic left heart syndrome.
  • Segmentation-derived measurements included cardiothoracic ratio, cardiac axis, and fractional area change; for example, mean cardiac axis was increased in tetralogy of Fallot at 63° ± 16° (range, 54–80°; P = 0.007) versus a normal reference of 45° ± 20°.

Methods

  • Retrospective, de-identified fetal echocardiograms and second-trimester screening ultrasounds from 18- to 24-week fetuses, performed between 2000 and 2019 at UCSF and Boston Children’s Hospital; fetuses with clinically significant non-cardiac malformations were excluded, and gold-standard normal/CHD labels came from expert review plus electronic health-record codes for studies from 2012 onward.
  • The ensemble used a ResNet-based view classifier for five recommended views (3VT, 3VV, LVOT, A4C, ABDO) plus non-target images, per-view binary diagnostic classifiers, a rule-based composite diagnostic classifier, and modified U-Net segmentation models for cardiothoracic structures.
  • Evaluation used five independent test sets—FETAL-125, OB-125, OB-4000, BCH-400, and TWINS-10—with metrics including AUC, sensitivity, specificity, predictive values, F score, and Jaccard similarity; model performance was also compared with seven clinicians on a five-image-per-heart task in OB-125.

Limitations

  • The paper states that testing was retrospective, that OB-4000 came from a center with fetal screening experience, and that prospective testing on consecutive cases across multiple centers—including rural, community, and non-expert centers—is needed, along with more multiple gestations and imaging with non-cardiac malformations.
  • The paper also states that more training data are needed for specific lesion types and segmentation, and that further clinical validation of segmentation-derived fetal biometrics is required, especially for CHD lesions without established metrics.
  • Evident from the text: the external BCH-400 test set had a very high CHD prevalence (92%) and lower composite AUC (0.89); the clinician comparison was limited to the 38 of 125 OB-125 surveys containing all five views; and data are not publicly available, with model weights available only upon request.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1101/2020.06.22.20137786

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

This check didn’t finish.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Arnaout et al. found an AUC of 0.99, a sensitivity of 95%, and specificity of 96% for detecting various cardiac views and distinguishing between normal and abnormal images on fetal echocardiogram.

    “Arnaout et al. used trained neural networks to detect the various cardiac views and distinguish between normal and abnormal images on fetal echocardiogram.77 The authors found an AUC of 0.99, a sensitivity of 95%, and specificity of 96%.”

    From Artificial intelligence in obstetric ultrasound: A scoping review. (2023) · cites this paper as [77]

    The Results section (“Classification of normal versus complex CHD”) reports an AUC of 0.99, sensitivity of 95%, and specificity of 96% for distinguishing normal from abnormal hearts in OB-4000. Those figures apply to fetal surveys, not fetal echocardiograms; the paper reports an AUC of 0.98 for the FETAL-125 echocardiogram set. View classification is a separate task with separately reported performance. (LLM, conf=0.90)

    Experimental — accuracy not yet measured
  • The combination of standard views allows for detection of up to 90% of CHD; however, in clinical practice, the detection rate is only about 30%.

    “The combination of standard views allows for the detection of up to 90% of CHD; however, in clinical practice, the detection rate is only about 30% [48,50].”

    From Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology (2023) · cites this paper as [50]

    The Abstract (page not specified) says the five recommended cardiac views could allow diagnosis of up to 90% of complex CHD, and that detection in practice is often as low as 30%. The citing sentence broadens “complex CHD” to all CHD and presents an occasional low detection rate as a general rate. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • AI applications rather assist than replace human observers.

    “summarized that AI applications rather assist than replace human observers [50].”

    From Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology (2023) · cites this paper as [50]

    The Discussion (page unavailable) proposes using the model to improve real-time ultrasound acquisition and help users obtain adequate views, supporting an assistive role. It does not conclude that AI applications generally assist rather than replace human observers; it also proposes standalone software that provides diagnostic predictions. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • AI-assisted US improved the accuracy of uterine fibroid assessment of young sonographers.

    “The retrospective study of Huo et al. showed that AI-assisted US improved the accuracy of uterine fibroid assessment of young sonographers”

    From Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology (2023) · cites this paper as [50]

    The Methods (Datasets) and Results (Classification of normal versus complex CHD; page not provided) concern fetal-heart ultrasound and detection of congenital heart disease. Searches for uterine fibroids, young sonographers, and Huo found no passage addressing the claimed improvement in fibroid assessment. (LLM, conf=0.78)

    Experimental — accuracy not yet measured
  • Previous research using DL methods from fetal ultrasound images can help increase the detection of fetal heart abnormalities compared to human experts.

    “The results obtained following previous research using the DL method from fetal US images can help increase the detection of fetal heart abnormalities compared to human experts [10,17,18,24].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Different lower confidence limits for OB-4000 sensitivity

    Minor

    The abstract and Results report the same 95% sensitivity and 96% specificity for the 4,108-survey test set, but give different lower limits for the sensitivity confidence interval: 84% and 83%.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [44]World Health Organization. WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening (2019). 31162915

Cited by

Sorted by citations

Authors
Arnaout, R., Curran, L., Zhao, Y., Levine, J. C., Chinn, E., & Moon-Grady, A. J.
Affiliations
  • University of California, San Francisco (US)
  • Boston Children's Hospital (US)
Corresponding author
Arnaout, Rima
Venue
Nature Medicine
Publisher
openRxiv
Published
2020-06-24
Volume
27
Issue
5
Pages
882-891
ISSN
1078-8956 (ISSN-L)
OpenAlex
W3161999894
Cited by
315
References
51
Citations by year
2026: 55 · 2025: 61 · 2024: 66 · 2023: 60 · 2022: 45 · 2021: 26 · 2020: 2
FWCI
14.24
Type
journal article
Language
en
Open access
Locations
  • Nature Medicine (publishedVersion)
  • medRxiv (acceptedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • eScholarship (California Digital Library) (submittedVersion)
Funders
  • U.S. Department of Defense (W81XWH1910294)
  • U.S. Department of Health and Human Services (R01HL150394)
  • American Heart Association (17IGMV33870001)
  • National Institutes of Health (TR991872, UL1 TR991872)
  • University of California, San Francisco
  • National Heart, Lung, and Blood Institute (R01HL150394)
  • National Center for Advancing Translational Sciences (UL1 TR991872)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Epidemiology › Congenital Heart Disease Studies
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
complex congenital heart disease, prenatal diagnosis, fetal echocardiography, fetal ultrasound screening, neural network ensembles, sensitivity and specificity
MeSH terms
Adult, Biometry, Female, Fetus, Heart, Heart Defects, Congenital, Humans, Mass Screening, Myocardium, Pregnancy, Pregnancy Trimester, Second, Prenatal Diagnosis, Sensitivity and Specificity, Thorax, Ultrasonography, Prenatal, Neural Networks, Computer, Echocardiography, Three-Dimensional, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review