Evaluation and Comparison of Anatomical Landmark Detection Methods for Cephalometric X-Ray Images: A Grand Challenge
Abstract
Cephalometric analysis is an essential clinical and research tool in orthodontics for the orthodontic analysis and treatment planning. This paper presents the evaluation of the methods submitted to the Automatic Cephalometric X-Ray Landmark Detection Challenge, held at the IEEE International Symposium on Biomedical Imaging 2014 with an on-site competition. The challenge was set to explore and compare automatic landmark detection methods in application to cephalometric X-ray images. Methods were evaluated on a common database including cephalograms of 300 patients aged six to 60 years, collected from the Dental Department, Tri-Service General Hospital, Taiwan, and manually marked anatomical landmarks as the ground truth data, generated by two experienced medical doctors. Quantitative evaluation was performed to compare the results of a representative selection of current methods submitted to the challenge. Experimental results show that three methods are able to achieve detection rates greater than 80% using the 4 mm precision range, but only one method achieves a detection rate greater than 70% using the 2 mm precision range, which is the acceptable precision range in clinical practice. The study provides insights into the performance of different landmark detection approaches under real-world conditions and highlights achievements and limitations of current image analysis techniques.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Manual labeling of landmarks is time-consuming, interrupts clinical workflow, and is subjective, yielding rater-dependent results.
“Manual labeling is time-consuming, interrupts the clinical workflow, and is subjective, which yields rater dependent results.”
From Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views (2019) · cites this paper as [28]
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300 · see citing works
Self-citationMultiscale vessel enhancement filtering
Frangi, A. F., Niessen, W. J., Vincken, K. L., & Viergever, M. A. (1998) · Lecture notes in computer science · doi:10.1007/bfb0056195 · Cited by 3,916 · see citing works
Pictorial Structures for Object Recognition
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000042934.15159.49 · Cited by 2,167 · see citing works
Ojala, T., Pietikäinen, M., & Harwood, D. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1994.576366 · Cited by 1,407 · see citing works
Cephalometric analysis for diagnosis and treatment of obstructive sleep apnea
Deberry‐Borowiecki, B., Kukwa, A., & Blanks, R. H. I. (1988) · The Laryngoscope · doi:10.1288/00005537-198802000-00021 · Cited by 319 · see citing works
Regression Forests for Efficient Anatomy Detection and Localization in CT Studies
Criminisi, A., Shotton, J., Robertson, D., & Ender Konukoglu, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-18421-5_11 · Cited by 275 · see citing works
Women and the Obstructive Sleep Apnea Syndrome
Guilleminault, C., Quera-Salva, M.-A., Partinen, M., & Jamieson, A. O. (1988) · CHEST Journal · doi:10.1378/chest.93.1.104 · Cited by 236 · see citing works
de Oliveira, A. E. F., Cevidanes, L., Phillips, C. L., et al. (2008) · Oral Surgery Oral Medicine Oral Pathology Oral Radiology and Endodontology · doi:10.1016/j.tripleo.2008.05.039 · Cited by 212 · see citing works
Fully Automatic Segmentation of the Proximal Femur Using Random Forest Regression Voting
Lindner, C., Thiagarajah, S., Wilkinson, J. M., The Consortium, Wallis, G. A., & Cootes, T. F. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2258030 · Cited by 198 · see citing works
Three-dimensional assessment of facial soft-tissue asymmetry before and after orthognathic surgery
Hajeer, M. Y., Ayoub, A. F., & Millett, D. T. (2004) · British Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.bjoms.2004.05.006 · Cited by 191 · see citing works
Kwon, T., Park, H., Ryoo, H., et al. (2005) · International Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.ijom.2005.04.006 · Cited by 175 · see citing works
Melnik, A. K. (1992) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/s0889-5406(05)80329-4 · Cited by 152 · see citing works
The clinical significance of error measurement in the interpretation of treatment results
Kamoen, A. (2001) · European Journal of Orthodontics · doi:10.1093/ejo/23.5.569 · Cited by 150 · see citing works
The esthetic impact of extraction and nonextraction treatments on Caucasian patients.
Bowman, S. J., & Johnston, L. E. (2000) · PubMed · doi:10.1043/0003-3219(2000)070 · Cited by 144 · see citing works
Surgical mandibular advancement: A cephalometric analysis of treatment response
Lake, S. L., McNeill, R., Little, R. M., et al. (1981) · American Journal of Orthodontics · doi:10.1016/0002-9416(81)90173-1 · Cited by 140 · see citing works
Cephalometric assessment of sagittal relationship between maxilla and mandible
Nanda, R. S., & Merill, R. M. (1994) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/s0889-5406(94)70127-x · Cited by 127 · see citing works
Hochban, W., & Brandenburg, U. (1994) · Journal of Cranio-Maxillofacial Surgery · doi:10.1016/s1010-5182(05)80559-1 · Cited by 123 · see citing works
Automated 2-D Cephalometric Analysis on X-ray Images by a Model-Based Approach
Yue, W., Yin, D., Li, C., et al. (2006) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2006.876638 · Cited by 111 · see citing works
Self-citationShape Representation for Efficient Landmark-Based Segmentation in 3-D
Ibragimov, B., Likar, B., Pernuš, F., & Vrtovec, T. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2296976 · Cited by 109 · see citing works
Self-citationAutomatic localization of craniofacial landmarks for assisted cephalometry
El-Feghi, I., Sid-Ahmed, M. A., & Ahmadi, M. A. (2003) · Pattern Recognition · doi:10.1016/j.patcog.2003.09.002 · Cited by 93 · see citing works
Advancement genioplasty: a retrospective cephalometric analysis of osseous and soft tissue changes.
McDonnell, J. P., McNeill, R., & West, R. A. (1977) · PubMed · Cited by 85 · see citing works
A Game-Theoretic Framework for Landmark-Based Image Segmentation
Ibragimov, B., Likar, B., Pernuš, F., & Vrtovec, T. (2012) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2012.2202915 · Cited by 61 · see citing works
Self-citationChen, C., Xie, W., Franke, J., Grützner, P.-A., Nolte, L.-P., & Guoyan Zheng, A. (2014) · Medical Image Analysis · doi:10.1016/j.media.2014.01.002 · Cited by 61 · see citing works
Self-citationSparse shape composition: A new framework for shape prior modeling
Zhang, S., Zhan, Y., Dewan, M., Huang, J., Metaxas, D., & Zhou, X. S. (2011) · doi:10.1109/cvpr.2011.5995322 · Cited by 55 · see citing works
Automatic landmarking of cephalograms using active appearance models
Vucinic, P., Trpovski, Ž., & Šćepan, I. (2010) · European Journal of Orthodontics · doi:10.1093/ejo/cjp099 · Cited by 52 · see citing works
The Accuracy of Cephalometric Tracing Superimposition
Gliddon, M. J., Xia, J. J., Gateño, J., et al. (2006) · Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.joms.2005.10.028 · Cited by 49 · see citing works
Automatic cephalometric landmark detection using Zernike moments and template matching
Kaur, A., & Singh, C. (2013) · Signal Image and Video Processing · doi:10.1007/s11760-013-0432-7 · Cited by 49 · see citing works
Ono, T., Lowe, A. A., Ferguson, K. A., & Fleetham, J. A. (1996) · PubMed · doi:10.1043/0003-3219(1996)066 · Cited by 47 · see citing works
Automatic Cephalometric X-Ray Landmark Detection by Applying Game Theory and Random Forests
Ibragimov, B. (2014) · Cited by 39 · see citing works
Self-citationDelamare, E. L., Liedke, G. S., Vizzotto, M. B., et al. (2010) · Dentomaxillofacial Radiology · doi:10.1259/dmfr/31924248 · Cited by 38 · see citing works
McClure, S. R., Sadowsky, P. L., Ferreira, A., et al. (2005) · Seminars in Orthodontics · doi:10.1053/j.sodo.2005.04.002 · Cited by 37 · see citing works
Evaluation of the reliability of computerized profile cephalometric analysis
Ferreira, J. T. L., & Telles, C. D. S. (2002) · Brazilian Dental Journal · doi:10.1590/s0103-64402002000300012 · Cited by 30 · see citing works
Bholsithi, W., Tharanon, W., Chintakanon, K., et al. (2009) · Biomedical Imaging and Intervention Journal · doi:10.2349/biij.5.4.e21 · Cited by 27 · see citing works
Seres, L., Varga, E., Kocsis, A., et al. (2014) · Head & Face Medicine · doi:10.1186/1746-160x-10-27 · Cited by 27 · see citing works
Automatic Localization of Interest Points in Zebrafish Images with Tree-Based Methods
Stern, O., Marée, R., Aceto, J., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24855-9_16 · Cited by 21 · see citing works
Discrimination of Bony Structures in Cephalograms for Automatic Landmark Detection
Kafieh, R., Sadri, S., Mehridehnavi, A., et al. (2008) · Communications in computer and information science · doi:10.1007/978-3-540-89985-3_75 · Cited by 14 · see citing works
Pereira, A. L., De‐Marchi, L. M., Scheibel, P. C., et al. (2010) · Cited by 8 · see citing works
Pereira, A. L., De‐Marchi, L. M., Scheibel, P. C., et al. (2010) · Dental Press Journal of Orthodontics · doi:10.1590/s2176-94512010000100008 · Cited by 5 · see citing works
Cephalometric X-Ray Registration using Angular Radial Transform
Kaur, A., & Singh, C. (2012) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ibragimov, B., & Xing, L. (2017) · Medical Physics · doi:10.1002/mp.12045 · Cited by 502 · see citing works
Self-citationLearning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views
Bier, B., Goldmann, F., Zaech, J.-N., Fotouhi, J., Hegeman, R., Grupp, R., Armand, M., Osgood, G., Navab, N., Maier, A., et al. (2019) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-019-01975-5 · Cited by 43 · see citing works
- Authors
- Wang, C.-W., Huang, C.-T., Hsieh, M.-C., Li, C.-H., Chang, S.-W., Li, W.-C., Vandaele, R., Maree, R., Jodogne, S., Geurts, P., Chen, C., Zheng, G., Chu, C., Mirzaalian, H., Hamarneh, G., Vrtovec, T., & Ibragimov, B.
- Affiliations
- National Taiwan University of Science and Technology (TW)
- Tri-Service General Hospital (TW)
- University of Liège (BE)
- University of Bern (CH)
- Simon Fraser University (CA)
- University of Ljubljana (SI)
- ORCID
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2015-09
- Volume
- 34
- Issue
- 9
- Pages
- 1890-1900
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W1995003188
- Cited by
- 235
- References
- 42
- Citations by year
- 2026: 26 · 2025: 28 · 2024: 39 · 2023: 29 · 2022: 33 · 2021: 29 · 2020: 14 · 2019: 15 · 2018: 9 · 2017: 5 · 2016: 5 · 2015: 3
- FWCI
- 9.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion)
- CentAUR (University of Reading) (submittedVersion)
- Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
- ORBi (University of Liège) (submittedVersion)
- Funders
- Cytomine research (n° 1017072)
- Ministry of Science and Technology (MOST-101-2628-E-011-006-MY3)
- Tri-Service General Hospital-National Taiwan University of Science and Technology Joint Research Program (TSGH-NTUST-102-02, TSGH-NTUST-103-02)
- Slovenian Research Agency (P2-0232 and L2-4072)
- FNRS-Télévie
- Topics
- Health Sciences › Dentistry › Oral Surgery › Dental Radiography and Imaging
- Health Sciences › Dentistry › Orthodontics › Orthodontics and Dentofacial Orthopedics
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- cephalometric landmark detection, cephalogram, automatic landmark detection, orthodontics, detection accuracy evaluation, ground truth, orthodontic treatment planning
- MeSH terms
- Adolescent, Adult, Cephalometry, Child, Head, Humans, Image Processing, Computer-Assisted, Middle Aged, Radiography, Dental, Young Adult, Anatomic Landmarks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
