3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model
Abstract
We present an accurate and robust framework for detecting and segmenting faces, localizing landmarks, and achieving fine registration of face meshes based on the fitting of a facial model. This model is based on a 3-D Point Distribution Model (PDM) that is fitted without relying on texture, pose, or orientation information. Fitting is initialized using candidate locations on the mesh, which are extracted from low-level curvature-based feature maps. Face detection is performed by classifying the transformations between model points and candidate vertices based on the upper-bound of the deviation of the parameters from the mean model. Landmark localization is performed on the segmented face by finding the transformation that minimizes the deviation of the model from the mean shape. Face registration is obtained using prior anthropometric knowledge and the localized landmarks. The performance of face detection is evaluated on a database of faces and non-face objects where we achieve an accuracy of 99.6%. We also demonstrate face detection and segmentation on objects with different scale and pose. The robustness of landmark localization is evaluated with noisy data and by varying the number of shapes and model points used in the model learning phase. Finally, face registration is compared with the traditional Iterative Closest Point (ICP) method and evaluated through a face retrieval and recognition framework on the GavabDB dataset, where we achieve a recognition rate of 87.4% and a retrieval rate of 83.9%.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
A Method for Registration of 3-D Shapes
Besl, P., & Mckay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 22,163 · see citing works
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190 · see citing works
Object modelling by registration of multiple range images
Chen, Y., & Medioni, G. G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,209 · see citing works
Iterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,557 · see citing works
Active Appearance Models Revisited
Matthews, I., & Baker, S. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029666.37597.d3 · Cited by 1,538 · see citing works
Schroeder, W. J., Zarge, J. A., & Lorensen, W. E. (1992) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/133994.134010 · Cited by 1,467 · see citing works
On Visual Similarity Based 3D Model Retrieval
Chen, D., Tian, X., Shen, Y., et al. (2003) · Computer Graphics Forum · doi:10.1111/1467-8659.00669 · Cited by 1,432 · see citing works
Object modeling by registration of multiple range images
Chen, Y., & Medioni, G. (2002) · Proceedings. 1991 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.1991.132043 · Cited by 1,409 · see citing works
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171 · see citing works
Robust registration of 2D and 3D point sets
Fitzgibbon, A. W. (2003) · Image and Vision Computing · doi:10.1016/j.imavis.2003.09.004 · Cited by 925 · see citing works
MESH: measuring errors between surfaces using the Hausdorff distance
Aspert, N., Santa-Cruz, D., & Ebrahimi, T. (2003) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2002.1035879 · Cited by 721 · see citing works
Gelfand, N., Mitra, N. J., Guibas, L., & Pottmann, H. (2005) · doi:10.2312/sgp/sgp05/197-206 · Cited by 545 · see citing works
An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition
Mian, A. S., Bennamoun, M., & Owens, R. A. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1105 · Cited by 469 · see citing works
Kakadiaris, I. A., Passalis, G., Toderici, G. D., et al. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1017 · Cited by 446 · see citing works
COSMOS-A representation scheme for 3D free-form objects
Dorai, C., & Jain, A. K. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.625113 · Cited by 438 · see citing works
Real-time combined 2D+3D active appearance models
Xiao, J., Baker, S. J., Matthews, I. A., et al. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315210 · Cited by 420 · see citing works
Towards a general multi-view registration technique
Bergevin, R., Soucy, M., Gagnon, H., et al. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.494643 · Cited by 410 · see citing works
Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression
Chang, K. I., Bowyer, K. W., & Flynn, P. J. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.210 · Cited by 362 · see citing works
Fully Automatic Registration of 3D Point Clouds
Makadia, A., Patterson, A., & Daniilidis, K. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.122 · Cited by 312 · see citing works
Face recognition based on depth and curvature features
Gordon, G. G. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1992.223253 · Cited by 259 · see citing works
Facial feature detection and face recognition from 2D and 3D images
Wang, Y., Chua, C.-S., & Ho, Y.-K. (2002) · Pattern Recognition Letters · doi:10.1016/s0167-8655(02)00066-1 · Cited by 250 · see citing works
3D face detection using curvature analysis
Colombo, A., Cusano, C., & Schettini, R. (2006) · Pattern Recognition · doi:10.1016/j.patcog.2005.09.009 · Cited by 229 · see citing works
Yamany, S. M., & Farag, A. A. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1023806 · Cited by 176 · see citing works
Robust Registration of 2D and 3D Point Sets
Fitzgibbon, A. W. (2001) · British Machine Vision Conference (BMVC) · doi:10.5244/c.15.43 · Cited by 148 · see citing works
How effective are landmarks and their geometry for face recognition?
Shi, J., Samal, A. K., & Marx, D. B. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2005.10.002 · Cited by 141 · see citing works
Adapting Active Shape Models for 3D Segmentation of Tubular Structures in Medical Images
de Bruijne, M., van Ginneken, B., Viergever, M. A., & Niessen, W. J. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45087-0_12 · Cited by 138 · see citing works
Shape-based 3D surface correspondence using geodesics and local geometry
Wang, Y., Peterson, B. S., & Staib, L. H. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.854933 · Cited by 133 · see citing works
Detection of Anchor Points for 3D Face Veri.cation
Colbry, D., Stockman, G. C., & Jain, A. (2006) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvpr.2005.441 · Cited by 118 · see citing works
Automatic Feature Extraction for Multiview 3D Face Recognition
Lu, X., & Jain, A. K. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.23 · Cited by 116 · see citing works
A laser scanning system for the measurement of facial surface morphology
Moss, J. P., Linney, A. D., Grindrod, S. R., et al. (1989) · Optics and Lasers in Engineering · doi:10.1016/0143-8166(89)90036-5 · Cited by 109 · see citing works
Face recognition from 2D and 3D images using 3D Gabor filters
Wang, Y., & Chua, C. (2005) · Image and Vision Computing · doi:10.1016/j.imavis.2005.07.005 · Cited by 90 · see citing works
Face recognition using 3D local geometrical features: PCA vs. SVM
Moreno, A. B., Sánchez, Á., Vélez, J. F., et al. (2005) · International symposium on image and signal processing and analysis/ISPA ... · doi:10.1109/ispa.2005.195407 · Cited by 74 · see citing works
AUTOMATIC 3D FACE VERIFICATION FROM RANGE DATA
Pun, G., & Wu, Z. (2003) · Cited by 72 · see citing works
Classifying range images of human faces with Hausdorff distance
Achermann, B., & Bunke, H. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2000.906199 · Cited by 66 · see citing works
A Fast Multi-Modal Approach to Facial Feature Detection
Boehnen, C. B., & Russ, T. D. (2005) · doi:10.1109/acvmot.2005.5 · Cited by 50 · see citing works
Matching 2.5D Scans for Face Recognition
Lu, X., Colbry, D., & Jain, A. K. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-25948-0_5 · Cited by 47 · see citing works
A Novel Method for 3D Face Detection and Normalization
Niese, R., Al-Hamadi, A. K., & Michaelis, B. (2007) · Journal of Multimedia · doi:10.4304/jmm.2.5.1-12 · Cited by 38 · see citing works
A Measure for Evaluating Retrieval Techniques based on Partially Ordered Ground Truth Lists
Typke, R., Veltkamp, R. C., & Wiering, F. (2006) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2006.262900 · Cited by 37 · see citing works
Automatic 3D face verification from range data
Pan, G., Wu, Z., & Pan, Y. (2003) · doi:10.1109/icme.2003.1221266 · Cited by 35 · see citing works
Automated Registration of 3D Faces using Dense Surface Models
Hutton, T. J., Buxton, B. F., & Hammond, P. (2003) · British Machine Vision Conference (BMVC) · doi:10.5244/c.17.45 · Cited by 33 · see citing works
A novel framework for automated 3D PDM construction using deformable models
Zhao, Z., & Teoh, E. K. (2005) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.594736 · Cited by 21 · see citing works
A New Method for Automatic 3D Face Registration
Ayyagari, V. R., Boughorbel, F., Koschan, A. F., et al. (2006) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvpr.2005.392 · Cited by 20 · see citing works
Volumetric segmentation via 3D active shape models
Dickens, M. M., Gleason, S. S., & Sari‐Sarraf, H. (2003) · IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) · doi:10.1109/iai.2002.999927 · Cited by 12 · see citing works
Region Segmentation and Feature Point Extraction on 3D Faces using a Point Distribution Model
Nair, P., & Cavallaro, A. (2007) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2007.4379252 · Cited by 11 · see citing works
Self-citationReconstruction of 3D tooth images
Buchaillard, S., Ong, S. H., Payan, Y., et al. (2005) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2004.1419489 · Cited by 10 · see citing works
Nair, P., Zou, L., & Cavallaro, A. (2005) · doi:10.1049/ic.2005.0714 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Zhao, X., Dellandréa, E., Zou, J., & Chen, L. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.001 · Cited by 27 · see citing works
- Authors
- Nair, P., & Cavallaro, A.
- Affiliations
- University of London (GB)
- ORCID
- Cavallaro, A.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2009-06
- Volume
- 11
- Issue
- 4
- Pages
- 611-623
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2128258626
- Cited by
- 135
- References
- 46
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 3 · 2022: 6 · 2021: 3 · 2020: 8 · 2019: 12 · 2018: 14 · 2017: 6 · 2016: 13 · 2015: 13 · 2014: 14 · 2013: 13 · 2012: 14
- FWCI
- 9.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Funders
- Research Foundation for the State University of New York
- Binghamton University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- face segmentation, facial landmark detection, point distribution model, facial shape modeling, face recognition, face retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review