What the human face tells the human mind: some challenges for the robot-human interface
Abstract
The human face serves a variety of different communicative functions in social interaction. The face mediates person identification, the perception of emotional expressions and lipreading. Perceiving the direction of social attention, and facial attractiveness, also affect interpersonal behaviour. This paper reviews these different uses made of facial information, and considers their computational demands. The possible link between the perception of faces and deeper levels of social understanding is emphasised through a discussion of developmental deficits affecting social cognition. Finally, the implications for the development of communication between robots and humans is discussed. It is concluded that it could be useful both for robots to understand human faces, and also to display human-like facial gestures themselves.>
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Lyons et al. method achieved a generalization rate of 92 percent when tested on 193 images divided into 10 segments.
“The generalization rate was 92 percent.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [51]
The Lyons et al. method achieved a generalization rate of 75 percent for recognition of expression of a novel subject when the image set was partitioned into nine segments, each corresponding to one expresser.
“The generalization rate was 75 percent for recognition of expression of a novel subject.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [51]
Lyons et al. used a similar face representation for expression classification into the six basic plus neutral emotion categories.
“A similar face representation was recently used by Lyons et al. [51] for expression classification into the six basic plus ªneutralº emotion categories.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [51]
Cites
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733 · see citing works
Does the autistic child have a “theory of mind” ?
Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985) · Cognition · doi:10.1016/0010-0277(85)90022-8 · Cited by 8,599 · see citing works
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · see citing works
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509 · see citing works
From Neuropsychology to Mental Structure
Shallice, T. (1988) · doi:10.1017/cbo9780511526817 · Cited by 4,072 · see citing works
Understanding face recognition
Bruce, V., & Young, A. (1986) · British Journal of Psychology · doi:10.1111/j.2044-8295.1986.tb02199.x · Cited by 3,900 · see citing works
Self-citationAn Experimental Study of Apparent Behavior
Heider, F., & Simmel, M. (1944) · The American Journal of Psychology · doi:10.2307/1416950 · Cited by 3,340 · see citing works
The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding
EKMAN, P., & FRIESEN, W. V. (1969) · Semiotica · doi:10.1515/semi.1969.1.1.49 · Cited by 3,316 · see citing works
Imitation of Facial and Manual Gestures by Human Neonates
Meltzoff, A. N., & Moore, M. (1977) · Science · doi:10.1126/science.198.4312.75 · Cited by 3,277 · see citing works
(1989) · UCL Discovery (University College London) · Cited by 2,724 · see citing works
Low-dimensional procedure for the characterization of human faces
Sirovich, L., & Kirby, M. J. L. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.000519 · Cited by 2,329 · see citing works
The Second Self: Computers and the Human Spirit
Kaplan, B., & Turkle, S. (1986) · Technology and Culture · doi:10.2307/3105353 · Cited by 1,721 · see citing works
The discrimination of speech sounds within and across phoneme boundaries.
Liberman, A. M., Harris, K. S., Hoffman, H. S., & Griffith, B. C. (1957) · Journal of Experimental Psychology · doi:10.1037/h0044417 · Cited by 1,692 · see citing works
Gaze and eye contact: a research review
Kleinke, C. L. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.100.1.78 · Cited by 1,455 · see citing works
Michotte, A. (2017) · Routledge eBooks · doi:10.4324/9781315519050 · Cited by 1,311 · see citing works
The capacity for joint visual attention in the infant
Scaife, M., & Bruner, J. S. (1975) · Nature · doi:10.1038/253265a0 · Cited by 1,158 · see citing works
Visual Following and Pattern Discrimination of Face-like Stimuli by Newborn Infants
Goren, C. C., Sarty, M., & Wu, P. Y. K. (1975) · Pediatrics · doi:10.1542/peds.56.4.544 · Cited by 1,158 · see citing works
Anderson, A. H., Bader, M., Bard, E. G., Boyle, E. A., Doherty, G. K., Garrod, S., Isard, S. D., Kowtko, J. C., McAllister, J., Miller, J., et al. (1991) · Language and Speech · doi:10.1177/002383099103400404 · Cited by 1,044 · see citing works
The second self: Computers and the human spirit
Taylor, W. D., & Damarin, S. K. (1985) · ECTJ · doi:10.1007/bf02769164 · Cited by 1,041 · see citing works
Defining the social deficits of autism: the contribution of non-verbal communication measures
Mundy, P., Sigman, M., Ungerer, J., & Sherman, T. (1986) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1986.tb00190.x · Cited by 956 · see citing works
Biology and cognitive development: the case of face recognition
(1992) · Choice Reviews Online · doi:10.5860/choice.30-0600 · Cited by 892 · see citing works
Organization and functions of cells responsive to faces in the temporal cortex
Perrett, D. I., Hietanen, J. K., Oram, M. W., & Benson, P. J. (1992) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1992.0003 · Cited by 882 · see citing works
Discrimination and imitation of facial expression by neonates
Field, T. M., Woodson, R., Greenberg, R., & Cohen, D. (1982) · Science · doi:10.1126/science.7123230 · Cited by 864 · see citing works
Perceptual role taking and protodeclarative pointing in autism
Baron‐Cohen, S. (1989) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.1989.tb00793.x · Cited by 733 · see citing works
Condon, W. S., & Sander, L. W. (1974) · Science · doi:10.1126/science.183.4120.99 · Cited by 729 · see citing works
Understanding face recognition with an interactive activation model
Burton, A. M., Bruce, V., & Johnston, R. (1990) · British Journal of Psychology · doi:10.1111/j.2044-8295.1990.tb02367.x · Cited by 609 · see citing works
Self-citationNeonatal recognition of the mother's face
Bushneil, I. W. R., Sai, F., & Mullin, J. (1989) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.1989.tb00784.x · Cited by 590 · see citing works
Categorical perception of facial expressions.
Etcoff, N., & Magee, J. J. (1992) · Cognition · doi:10.1016/0010-0277(92)90002-y · Cited by 587 · see citing works
Siegel, J. M. (1990) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.58.6.1081 · Cited by 483 · see citing works
Siegel, J. M. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.58.6.1081 · Cited by 483 · see citing works
Neuropsychological, neurological, and neuroanatomical profile of Williams syndrome
Bellugi, U., Bihrle, A. M., Jernigan, T. L., Trauner, D., & Doherty, S. (2005) · American Journal of Medical Genetics · doi:10.1002/ajmg.1320370621 · Cited by 423 · see citing works
Sex Discrimination: How Do We Tell the Difference between Male and Female Faces?
Bruce, V., Burton, A. M., Hanna, E., et al. (1993) · Perception · doi:10.1068/p220131 · Cited by 422 · see citing works
Self-citationSome preliminaries to a comprehensive account of audio-visual speech perception.
Summerfield, Q. A. (1987) · Cited by 417 · see citing works
Perrett, D. I., Smith, P., Potter, D. D., Mistlin, A. J., Head, A., Milner, A. D., & Jeeves, M. A. (1984) · PubMed · Cited by 404 · see citing works
What's the Difference between Men and Women? Evidence from Facial Measurement
Burton, A. M., Bruce, V., & Dench, N. (1993) · Perception · doi:10.1068/p220153 · Cited by 367 · see citing works
Self-citationThe induced affect response: 10-week-old infants' responses to three emotion expressions.
Haviland, J. M., & Lelwica, M. (1987) · Developmental Psychology · doi:10.1037/0012-1649.23.1.97 · Cited by 361 · see citing works
Lipreading and audio-visual speech perception
Summerfield, A. Q. (1992) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1992.0009 · Cited by 359 · see citing works
Towards a Mechanism of Joint Visual Attention in Human Infancy
Butterworth, G. E., & Cochran, E. L. (1980) · International Journal of Behavioral Development · doi:10.1177/016502548000300303 · Cited by 350 · see citing works
The Social Psychology of Facial Appearance
Bull, R. H. C., & Rumsey, N. (1988) · Springer eBooks · doi:10.1007/978-1-4612-3782-2 · Cited by 347 · see citing works
Hearing by eye—the psychology of lip reading
Montgomery, G. W. G. (1988) · Behaviour Research and Therapy · doi:10.1016/0005-7967(88)90161-1 · Cited by 272 · see citing works
Sensitivity to eye gaze in prosopagnosic patients and monkeys with superior temporal sulcus ablation
Campbell, R., Heywood, C. A., Cowey, A., Regard, M., & Landis, T. (1990) · Neuropsychologia · doi:10.1016/0028-3932(90)90050-x · Cited by 254 · see citing works
Green, K. P., Kuhl, P. K., Meltzoff, A. N., & Stevens, E. B. (1991) · Perception & Psychophysics · doi:10.3758/bf03207536 · Cited by 239 · see citing works
Face recognition and lip-reading in autism
Gelder, B. D., Vroomen, J., & van der Heide, L. (1991) · European Journal of Cognitive Psychology · doi:10.1080/09541449108406220 · Cited by 235 · see citing works
Automatic extraction of face-features
Craw, I., Ellis, H., & Lishman, J. R. (1987) · Pattern Recognition Letters · doi:10.1016/0167-8655(87)90039-0 · Cited by 195 · see citing works
Matching familiar and unfamiliar faces on identity and expression
Young, A. W., McWeeny, K. H., Hay, D. C., & Ellis, A. W. (1986) · Psychological Research · doi:10.1007/bf00309318 · Cited by 191 · see citing works
FACE RECOGNITION AND LIPREADING
Campbell, R., Landis, T., & Regard, M. (1986) · Brain · doi:10.1093/brain/109.3.509 · Cited by 185 · see citing works
I recognize your face but I can't remember your name: A simple explanation?
Burton, A. M., & Bruce, V. (1992) · British Journal of Psychology · doi:10.1111/j.2044-8295.1992.tb02424.x · Cited by 172 · see citing works
Self-citationIntermodal perception of expressive behaviors: Relation of eye and voice?
Walker‐Andrews, A. S. (1986) · Developmental Psychology · doi:10.1037/0012-1649.22.3.373 · Cited by 171 · see citing works
The role of the ‘face-cell’ area in the discrimination and recognition of faces by monkeys
Heywood, C. A., & Cowey, A. (1992) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1992.0004 · Cited by 170 · see citing works
Influences of Familiarity on the Processing of Faces
Bruce, V. (1986) · Perception · doi:10.1068/p150387 · Cited by 142 · see citing works
Self-citationNormality and Pathology in Cognitive Function
Wyke, M. (1983) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp.46.2.193-a · Cited by 133 · see citing works
The Effect of Facial Disfigurement on the Proxemic Behavior of the General Public 1
Rumsey, N., Bull, R. H. C., & Gahagan, D. (1982) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1982.tb00855.x · Cited by 114 · see citing works
Practical Face Recognition and Verification with Wisard
Stonham, T. J. (1986) · Springer eBooks · doi:10.1007/978-94-009-4420-6_44 · Cited by 91 · see citing works
The Therapeutic Roles of Cat Mascots with a Hospital-based Geriatric Population: A Staff Survey
Brickel, C. M. (1979) · The Gerontologist · doi:10.1093/geront/19.4.368 · Cited by 85 · see citing works
Perceptual categories and the computation of “grandmother”
Young, A. W., & Bruce, V. (1991) · The European Journal of Cognitive Psychology · doi:10.1080/09541449108406218 · Cited by 76 · see citing works
Self-citationBruce, V., & Burton, A. M. (1992) · Medical Entomology and Zoology · Cited by 71 · see citing works
Self-citationBruce, V., Burton, A. M., & Craw, I. (1992) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1992.0015 · Cited by 64 · see citing works
Self-citationMorley, I. E., & Stephenson, G. M. (1969) · British Journal of Psychology · doi:10.1111/j.2044-8295.1969.tb01228.x · Cited by 61 · see citing works
The Relation of Interactional Synchrony to Cognitive and Emotional Processes
CONDON, W. S. (1980) · Mouton eBooks · doi:10.1515/9783110813098.49 · Cited by 58 · see citing works
The Role of Visual Communication in Social Exchange
Stephenson, G. M., Ayling, K., & Rutter, D. R. (1976) · British Journal of Social and Clinical Psychology · doi:10.1111/j.2044-8260.1976.tb00016.x · Cited by 56 · see citing works
Bruce, V., & Burton, A. M. (1989) · Elsevier eBooks · doi:10.1016/b978-0-444-87143-5.50037-8 · Cited by 34 · see citing works
Self-citationThe computer synthesis of expressive faces
Waters, K., & Terzopoulos, D. (1992) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1992.0011 · Cited by 28 · see citing works
Effects of distinctiveness, repetition and semantic priming on the recognition of face familiarity.
Bruce, V., Dench, N., & Burton, A. M. (1993) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0078773 · Cited by 16 · see citing works
Self-citationEye‐contact in dyads: A test of the Independence Hypothesis
Lazzerini, A. J., Stephenson, G. M., & Neave, H. R. (1978) · British Journal of Social and Clinical Psychology · doi:10.1111/j.2044-8260.1978.tb00270.x · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · see citing works
- Authors
- Bruce, V.
- Affiliations
- University of Nottingham (GB)
- Corresponding author
- Bruce, V.
- ORCID
- Bruce, V.
- Venue
- [1992] Proceedings IEEE International Workshop on Robot and Human Communication
- Publisher
- IEEE
- Published
- 2003-01-02
- Pages
- 44-51
- ISSN
- 1944-9437 (ISSN-L)
- OpenAlex
- W1892190109
- Cited by
- 32
- References
- 64
- Citations by year
- 2020: 1 · 2019: 2 · 2018: 5 · 2017: 1 · 2016: 2 · 2015: 4 · 2014: 2 · 2013: 1 · 2012: 1
- FWCI
- 3.47
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE RO-MAN (publishedVersion)
- Funders
- University of Nottingham
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- face perception, social cognition, emotional expression recognition, lip reading, personal identification, social attention, facial attractiveness, facial gestures, human-robot communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review