Determining facial expressions in real time
Abstract
We suggest an approach to describing and tracking the deformation of facial features. We concentrate on the mouth since its shape is important in detecting emotion. However, we believe that our system could be extended to deal with other facial features. In our system, the mouth is described by a valley contour which is based between the lips. This contour is shown to exist independently of illumination, viewpoint, identity, and expression. We present a real time mouth tracking system that follows this valley. It is shown to be robust to changes in identity, illumination and viewpoint. A simple classification algorithm was found to be sufficient to discriminate between 5 different mouth shapes, with a 100% recognition rate.>
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For most work in automatic facial expression analysis, the conditions under which an image is obtained are controlled.
“For most of the work in automatic facial expression analysis, the conditions under which an image is obtained are controlled.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
Pantic and Rothkrantz's face detection system does not allow facial hair or glasses.
“No facial hair or glasses are allowed.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
In most work in automatic facial expression analysis, the camera is either mounted on a helmetlike device worn by the subject or placed so that the image has the face in frontal view.
“The camera is either mounted on a helmetlike device worn by the subject (e.g., [62], [59]) or placed in such a way that the image has the face in frontal view.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
Pantic and Rothkrantz's system cannot deal with minor inaccuracies of the extracted facial data and deals merely with images of faces without facial hair or glasses.
“The system cannot deal with minor inaccuracies of the extracted facial data and it deals merely with images of faces without facial hair or glasses.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
The performance of Pantic and Rothkrantz's detection scheme was tested on 496 dual views, and human observers in 89 percent approved when visually inspected the achieved localization of the facial features.
“The performance of the detection scheme was tested on 496 dual views.Human observers in 89 percent approved when visually inspected the achieved localization of the facial features.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
The overall performance of the Pantic and Rothkrantz system achieved a correct recognition ratio of 91 percent on a set of 265 dual facial views representing six basic and various blended emotional expressions shown by eight subjects.
“A correct recognition ratio of 91 percent has been reported.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
Only two of the surveyed systems perform quantified facial expression classification into multiple basic emotion categories.
“Only two of the surveyed systems ( [62], [106]) perform quantified facial expression classification into multiple basic emotion categories.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
The dual-views used for testing of the Pantic and Rothkrantz system have been recorded under constant illumination and none of the subjects had a moustache, a beard, or wear glasses.
“The dual-views used for testing of the system have been recorded under constant illumination and none of the subjects had a moustache, a beard, or wear glasses.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
Only two of the surveyed systems process images acquired by a mounted camera.
“Only two of the surveyed systems process images acquired by a mounted camera ( [59], [62])”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
The Pantic and Rothkrantz system achieved an average recognition rate of 92 percent for the upper face AUs and 86 percent for the lower face AUs.
“The average recognition rate was 92 percent for the upper face AUs and 86 percent for the lower face AUs.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [62]
Cites
Bar‐Shalom, Y., Fortmann, T. E., & Cable, P. G. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.398863 · Cited by 3,080 · see citing works
Bar‐Shalom, Y. (1988) · Medical Entomology and Zoology · Cited by 3,067 · see citing works
A muscle model for animation three-dimensional facial expression
Waters, K. (1987) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/37401.37405 · Cited by 744 · see citing works
Feature extraction from faces using deformable templates
Yuille, A. L., Cohen, D. S., & Hallinan, P. W. (2003) · Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1989.37836 · Cited by 708 · see citing works
The variational approach to shape from shading
Horn, B. K. P., & Brooks, M. J. (1986) · Computer Vision Graphics and Image Processing · doi:10.1016/0734-189x(86)90114-3 · Cited by 596 · see citing works
Analysis and synthesis of facial image sequences using physical and anatomical models
Terzopoulos, D., & Waters, K. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.216726 · Cited by 538 · see citing works
A muscle model for animation three-dimensional facial expression
Waters, K. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37405 · Cited by 476 · see citing works
Some preliminaries to a comprehensive account of audio-visual speech perception.
Summerfield, Q. A. (1987) · Cited by 417 · see citing works
Nugteren, C., Corporaal, H., Nugteren, C., et al. (2012) · SpringerReference · doi:10.1007/springerreference_178824 · Cited by 317 · see citing works
Determining the gaze of faces in images
Gee, A. H., & Cipolla, R. (1994) · Image and Vision Computing · doi:10.1016/0262-8856(94)90039-6 · Cited by 242 · see citing works
Learning to track the visual motion of contours
Blake, A., Isard, M., & Reynard, D. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(95)00032-1 · Cited by 182 · see citing works
Self-citationTricot, M. (1986) · Journal of Systems and Software · doi:10.1016/0164-1212(86)90027-0 · Cited by 164 · see citing works
Shepherd, J. W. (1981) · Medical Entomology and Zoology · Cited by 153 · see citing works
Hekster, O. J. (2002) · BRILL eBooks · doi:10.1163/9789004502321_009 · Cited by 137 · see citing works
3D position, attitude and shape input using video tracking of hands and lips
Blake, A., & Isard, M. (1994) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/192161.192197 · Cited by 90 · see citing works
Self-citationPearson, D. E., Hanna, E. Y., & Martinez, K. (1990) · ePrints Soton (University of Southampton) · Cited by 29 · see citing works
Towards the Quantification of Facial Expressions with the Use of a Mathematic Model of the Face
Pilowsky, I., Thornton, M., & Stokes, B. B. (1986) · Springer eBooks · doi:10.1007/978-94-009-4420-6_36 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · see citing works
- Authors
- Moses, Y., Reynard, D., & Blake, A.
- Affiliations
- University of Oxford (GB)
- ORCID
- Blake, A.
- Venue
- Proceedings of IEEE International Conference on Computer Vision
- Publisher
- IEEE Comput. Soc. Press
- Published
- 2002-11-19
- Pages
- 296-301
- OpenAlex
- W2112692809
- Cited by
- 51
- References
- 17
- Citations by year
- 2017: 2 · 2014: 4 · 2013: 2 · 2012: 1
- FWCI
- 3.61
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, mouth tracking, lip contour, emotion recognition, real-time tracking, illumination robustness, viewpoint robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
