Identifying facial phenotypes of genetic disorders using deep learning
Abstract
Syndromic genetic conditions, in aggregate, affect 8% of the population 1. Many syndromes have recognizable facial features 2 that are highly informative to clinical geneticists 3-5. Recent studies show that facial analysis technologies measured up to the capabilities of expert clinicians in syndrome identification 6-9. However, these technologies identified only a few disease phenotypes, limiting their role in clinical settings, where hundreds of diagnoses must be considered. Here we present a facial image analysis framework, DeepGestalt, using computer vision and deep-learning algorithms, that quantifies similarities to hundreds of syndromes. DeepGestalt outperformed clinicians in three initial experiments, two with the goal of distinguishing subjects with a target syndrome from other syndromes, and one of separating different genetic subtypes in Noonan syndrome. On the final experiment reflecting a real clinical setting problem, DeepGestalt achieved 91% top-10 accuracy in identifying the correct syndrome on 502 different images. The model was trained on a dataset of over 17,000 images representing more than 200 syndromes, curated through a community-driven phenotyping platform. DeepGestalt potentially adds considerable value to phenotypic evaluations in clinical genetics, genetic testing, research and precision medicine.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Facial analysis technology has proven effective in diagnosing TS in resource limited countries.
“In resource limited countries, facial analysis technology has proven effective in diagnosing TS. Kruszka et al.99 used the DeepGestalt model100 to differentiate TS from unaffected controls and controls with Noonan syndrome in diverse populations.”
From Clinical practice guidelines for the care of girls and women with Turner syndrome. (2024) · cites this paper as [100]
Cites
Advances in neural information processing systems 7
Cios, K. J., & Shields, M. E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,089 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · see citing works
(2010) · Cited by 14,134 · see citing works
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · see citing works
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · see citing works
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445 · see citing works
A convolutional neural network cascade for face detection
Li, H., Lin, Z., Shen, X., Brandt, J., & Hua, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299170 · Cited by 1,390 · see citing works
Clinical Diagnostics in Human Genetics with Semantic Similarity Searches in Ontologies
Köhler, S., Schulz, M. H., Krawitz, P., Bauer, S., Dölken, S., Ott, C. E., Mundlos, C., Horn, D., Mundlos, S., & Robinson, P. N. (2009) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2009.09.003 · Cited by 590 · see citing works
Self-citationGenetic disorders in children and young adults: a population study.
Baird, P. A., Anderson, T., Newcombe, H. B., & Lowry, R. B. (1988) · Europe PMC (PubMed Central) · Cited by 402 · see citing works
Proceedings of the British Machine Vision Conference 2021
Rajpoot, N., & Bhalerao, A. H. (2021) · doi:10.5244/c.35 · Cited by 393 · see citing works
Robinson, P. N., & Mundlos, S. (2010) · Clinical Genetics · doi:10.1111/j.1399-0004.2010.01436.x · Cited by 357 · see citing works
Learning to Align from Scratch
Huang, G. B., Mattar, M., Lee, H., & Miller, E. G. (2012) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 270 · see citing works
Web-scale training for face identification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298891 · Cited by 232 · see citing works
The London Dysmorphology Database.
Winter, R. M., & Baraitser, M. (1987) · Journal of Medical Genetics · doi:10.1136/jmg.24.8.509 · Cited by 163 · see citing works
Diagnostically relevant facial gestalt information from ordinary photos
Ferry, Q. R. V., Steinberg, J., Webber, C., FitzPatrick, D. R., Ponting, C. P., Zisserman, A., & Nellåker, C. (2014) · eLife · doi:10.7554/elife.02020 · Cited by 161 · see citing works
Next-generation sequencing demands next-generation phenotyping
Hennekam, R. C. M., & Biesecker, L. G. (2012) · Human Mutation · doi:10.1002/humu.22048 · Cited by 158 · see citing works
Zenker, M., Horn, D., Wieczorek, D., Allanson, J. E., Pauli, S., van der Burgt, I., Doerr, H. G., Gaspar, H., Hofbeck, M., Gillessen‐Kaesbach, G., et al. (2007) · Journal of Medical Genetics · doi:10.1136/jmg.2007.051276 · Cited by 138 · see citing works
Self-citationGenotype and phenotype in patients with Noonan syndrome and a RIT1 mutation
Kouz, K., Lißewski, C., Spranger, S., Mitter, D., Rieß, A., López‐González, V., Lüttgen, S., Aydın, H., von Deimling, F., Evers, C., et al. (2016) · Genetics in Medicine · doi:10.1038/gim.2016.32 · Cited by 109 · see citing works
Self-citationFacial diagnosis of mild and variant CdLS: Insights from a dysmorphologist survey
Rohatgi, S., Clark, D. M., Kline, A. D., Jackson, L. G., Pié, J., Siu, V. M., Ramos, F. J., Krantz, I. D., & Deardorff, M. A. (2010) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.33441 · Cited by 91 · see citing works
Genetic studies of craniofacial anomalies: clinical implications and applications
Hart, T., & Hart, P. S. (2009) · Orthodontics and Craniofacial Research · doi:10.1111/j.1601-6343.2009.01455.x · Cited by 90 · see citing works
Recognition of the Cornelia de Lange syndrome phenotype with facial dysmorphology novel analysis
Basel‐Vanagaite, L., Wolf, L., Orin, M. J., Larizza, L., Gervasini, C., Krantz, I. D., & Deardoff, M. A. (2015) · Clinical Genetics · doi:10.1111/cge.12716 · Cited by 89 · see citing works
The face of Noonan syndrome: Does phenotype predict genotype
Allanson, J. E., Bohring, A., Dörr, H., et al. (2010) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.33518 · Cited by 75 · see citing works
A Deep Learning Frame-Work for Recognizing Developmental Disorders
Shukla, P., Gupta, T., Saini, A., Singh, P., & Raman, B. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.84 · Cited by 73 · see citing works
Natural history of Sanfilippo syndrome in Spain
Delgadillo, V., O’Callaghan, M. D. M., Gort, L., et al. (2013) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-8-189 · Cited by 72 · see citing works
Zarate, Y. A., Smith‐Hicks, C. L., Greene, C., Abbott, M., Siu, V. M., Calhoun, A. R. U. L., Pandya, A., Li, C., Sellars, E. A., Kaylor, J., et al. (2018) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.38630 · Cited by 71 · see citing works
Computer-Aided Recognition of Facial Attributes for Fetal Alcohol Spectrum Disorders
Valentine, M. J., Bihm, D. C. J., Wolf, L., et al. (2017) · PEDIATRICS · doi:10.1542/peds.2016-2028 · Cited by 60 · see citing works
Liehr, T., Acquarola, N., Pyle, K., St‐Pierre, S., Rinholm, M., Bar, O., Wilhelm, K., & Schreyer, I. (2017) · Clinical Genetics · doi:10.1111/cge.13087 · Cited by 48 · see citing works
Self-citationAutomatic recognition of the XLHED phenotype from facial images
Hadj-Rabia, S., Schneider, H., Navarro, E., Klein, O. D., Kirby, N., Huttner, K. M., Wolf, L., Orin, M. J., Wohlfart, S., Bodemer, C., et al. (2017) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.38343 · Cited by 47 · see citing works
Using Linking Features in Learning Non-parametric Part Models
Karlinsky, L., & Ullman, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33712-3_24 · Cited by 46 · see citing works
Gripp, K. W., Baker, L., Telegrafi, A. B., & Monaghan, K. G. (2016) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.37672 · Cited by 46 · see citing works
Self-citationCavé, H., Caye‐Eude, A., Ghedira, N., Capri, Y., Pouvreau, N., Fillot, N., Trimouille, A., Vignal, C., Fenneteau, O., Alembik, Y., et al. (2016) · European Journal of Human Genetics · doi:10.1038/ejhg.2015.273 · Cited by 30 · see citing works
Using facial images for the diagnosis of genetic syndromes: A survey
Rai, M. C. E., Werghi, N., Muhairi, H. A., et al. (2015) · International Conference on Communications, Signal Processing, and their Applications (ICCSPA) · doi:10.1109/iccspa.2015.7081271 · Cited by 15 · see citing works
Novel mutations in PTPN11 gene in two girls with Noonan syndrome phenotype
Güleç, E. Y., Ocak, Z., Candan, Ş., et al. (2015) · International Journal of Cardiology · doi:10.1016/j.ijcard.2015.03.260 · Cited by 7 · see citing works
Genotype – Phenotype Correlations in Noonan Syndrome
Rusu, C. (2014) · Acta Endocrinologica (Bucharest) · doi:10.4183/aeb.2014.463 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136 · see citing works
Precision medicine, AI, and the future of personalized health care
Johnson, K. B., Wei, W., Weeraratne, D., Frisse, M. E., Misulis, K., Rhee, K., Zhao, J., & Snowdon, J. L. (2021) · Clinical and Translational Science · doi:10.1111/cts.12884 · Cited by 1,854 · see citing works
Therapies for rare diseases: therapeutic modalities, progress and challenges ahead
Tambuyzer, E., Vandendriessche, B., Austin, C. P., Brooks, P. J., Larsson, K., Miller Needleman, K. I., Valentine, J., Davies, K., Groft, S. C., Preti, R., et al. (2020) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0049-9 · Cited by 357 · see citing works
Armoundas, A. A., Narayan, S. M., Arnett, D. K., Spector-Bagdady, K., Bennett, D. A., Celi, L. A., Friedman, P. A., Gollob, M. H., Hall, J. L., Kwitek, A. E., et al. (2024) · Circulation · doi:10.1161/cir.0000000000001201 · Cited by 347 · see citing works
GestaltMatcher facilitates rare disease matching using facial phenotype descriptors
Hsieh, T.-C., Bar-Haim, A., Moosa, S., Ehmke, N., Gripp, K. W., Pantel, J. T., Danyel, M., Mensah, M. A., Horn, D., Rosnev, S., et al. (2022) · Nature Genetics · doi:10.1038/s41588-021-01010-x · Cited by 233 · see citing works
Self-citationClinical practice guidelines for the care of girls and women with Turner syndrome.
Gravholt, C. H., Andersen, N. H., Christin-Maitre, S., Davis, S. M., Duijnhouwer, A., Gawlik, A., Maciel-Guerra, A. T., Gutmark-Little, I., Fleischer, K., Hong, D., et al. (2024) · European Journal of Endocrinology · doi:10.1093/ejendo/lvae050 · Cited by 224 · see citing works
Artificial intelligence (AI) in rare diseases: is the future brighter?
Brasil, S., Pascoal, C., Francisco, R., dos Reis Ferreira, V., A. Videira, P., & Valadão, G. (2019) · Genes · doi:10.3390/genes10120978 · Cited by 160 · see citing works
Implementation of artificial intelligence in medicine: status analysis and development suggestions
Xiang, Y., Zhao, L., Liu, Z., Wu, X., Chen, J., Long, E., Lin, D., Zhu, Y., Chen, C., Lin, Z., et al. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101780 · Cited by 100 · see citing works
NEXMIF encephalopathy: an X-linked disorder with male and female phenotypic patterns
Stamberger, H., Hammer, T. B., Gardella, E., Vlaskamp, D. R. M., Bertelsen, B., Mandelstam, S., de Lange, I., Zhang, J., Myers, C. T., Fenger, C., et al. (2021) · Genetics in Medicine · doi:10.1038/s41436-020-00988-9 · Cited by 57 · see citing works
Tao, L., Zhang, C., Zeng, L., Zhu, S., Li, N., Li, W., Zhang, H., Zhao, Y., Zhan, S., & Ji, H. (2020) · JMIR Medical Informatics · doi:10.2196/16912 · Cited by 42 · see citing works
Perrier, E., Rifai, M., Terzic, A., Dubois, C., & Cohen, J. F. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.1065957 · Cited by 37 · see citing works
Ebstein, F., Küry, S., Most, V., Rosenfelt, C., Scott-Boyer, M.-P., van Woerden, G. M., Besnard, T., Papendorf, J. J., Studencka-Turski, M., Wang, T., et al. (2023) · Science Translational Medicine · doi:10.1126/scitranslmed.abo3189 · Cited by 36 · see citing works
Self-citationFew shot learning for phenotype-driven diagnosis of patients with rare genetic diseases
Alsentzer, E., Li, M. M., Kobren, S. N., Noori, A., Undiagnosed Diseases Network, Kohane, I. S., & Zitnik, M. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01749-1 · Cited by 54 · see citing works
Turner syndrome in diverse populations
Kruszka, P., Addissie, Y. A., Tekendo‐Ngongang, C., Jones, K. L., Savage, S. K., Gupta, N., Sirisena, N. D., Dissanayake, V. H. W., Paththinige, C. S., Aravena, T., et al. (2020) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.61461 · Cited by 27 · see citing works
Self-citationUsing deep learning and electronic health records to detect Noonan syndrome in pediatric patients
Yang, Z., Shikany, A., Ni, Y., Zhang, G., Weaver, K. N., & Chen, J. (2022) · Genetics in Medicine · doi:10.1016/j.gim.2022.08.002 · Cited by 20 · see citing works
Tang, J., Han, J., Xie, B., Xue, J., Zhou, H., Jiang, Y., Hu, L., Chen, C., Zhang, K., Zhu, F., et al. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20032377 · Cited by 13 · see citing works
- Authors
- Gurovich, Y., Hanani, Y., Bar, O., Nadav, G., Fleischer, N., Gelbman, D., Basel-Salmon, L., Krawitz, P. M., Kamphausen, S. B., Zenker, M., Bird, L. M., & Gripp, K. W.
- Affiliations
- University of Bonn (DE)
- University Hospital Magdeburg (DE)
- University of California San Diego (US)
- Alfred I. duPont Hospital for Children (US)
- Corresponding author
- Gurovich, Yaron
- ORCID
- Venue
- Nature Medicine
- Published
- 2019-01
- Volume
- 25
- Issue
- 1
- Pages
- 60-64
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W2905441841
- Cited by
- 739
- References
- 40
- Citations by year
- 2026: 74 · 2025: 96 · 2024: 94 · 2023: 117 · 2022: 111 · 2021: 89 · 2020: 97 · 2019: 60
- FWCI
- 30.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- facial phenotyping, rare genetic disorders, facial image analysis
- MeSH terms
- Deep Learning, Algorithms, Genotype, Humans, Image Processing, Computer-Assisted, Phenotype, Syndrome, Facies, Genetic Diseases, Inborn
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
