Understanding the recognition of facial identity and facial expression

“Is multimodal analysis involved? The co-occurrence of impairments in the recognition of facial and vocal expression is not restricted to cases that show disproportionate impairments for one emotion; in fact, the relationship is if anything more striking in patients who have general problems in recognizing facial expre…”

“…(identity, gender, age) and transient (facial affect, smile sincerity) attributes, suggesting that whole-face presentation aids the encoding of face structure at a relatively early stage of face processing (Bruce & Young, 1986; Calder & Young, 2005; Duchaine & Yovel, 2015; Haxby et al., 2000).”

“Based on the literature (e.g., Adolphs et al., 2005; Bassili, 1979; Calder and Young, 2005; Eisenbarth and Alpers, 2011), we concentrated on four discrete emotion types—fear, sadness, disgust, and happiness—for which there is some evidence that the location of informative facial features necessary to identify these em…”

“However, more recent studies have suggested that there is significant cross-talk between face identification and emotion processing (Schweinberger et al 1999; Ganel and Goshen-Gottstein 2004; Calder and Young 2005).”

“This is an important issue as some prosopagnosics also suffer from impairments in the perception of emotional expressions (Calder & Young, 2005 ; but see K.”

“Neuropsychological and, to a lesser degree, neuroimaging and electrophysiologial studies, show that facial expressions and facial identity are processed differently (e.g., Haxby, Hoffman, & Gobbini, 2000; Munte et al., 1998; Sergent, Ohta, MacDonald, & Zuck, 1994; although see Calder & Young, 2005 ).”

“…facial aspects (i.e., facial expression) [Allison et al., 2000; Engell and Haxby, 2007; Haxby et al., 2000], though there is also evidence suggesing relatvie rather than absolute dissociation of identity and expresssion processing [e.g., Calder, 2011; Calder and Young, 2005; Fox et al., 2009].”

“While patient-based evidence supporting the independence of facial identity and expression processing was almost entirely discounted by Calder and Young (2005), more recent neuropsychological case studies have provided more convincing evidence supporting the independence hypothesis, particularly…”

“Most cases of prosopagnosia, for example, show some evidence for impaired emotion processing as well (Calder and Young 2005).”

“The debate can be seen in that while one well-respected model of face perception suggests that the superior temporal sulcus (STS) processes facial expressions and other transient facial signals [4,5], other researchers [6] note that emotional displays are often audiovisual (AV) by nature and suggest the STS is an inte…”

“In other words, a facial expression can be roughly decomposed to an expressive component and neutral component [31, 3, 35].”

“We note that the P frames can also incorporate the relative position of face key points, which are also related to the identity [15, 73].”

“Neuropsychological studies using fMRI technology have provided conflicting information about the extent to which identity and facial expression are processed by separate neural substrates (see Haxby, Hoffman, & Gobbini, 2000 and Calder & Young, 2005 for contrasting views).”

“…Gobbini, 2000), behavioural (for a review, see Palermo & Rhodes, 2007) and neuroscience reports (e.g., Vuilleumier et al., 2003) that recognising the identity of a face and discriminating the emotion that it is expressing are quite distinct cognitive operations (although see Calder & Young, 2005).”

“emphasize the interaction of emotional expression and personal identity and support approaches that propose a relative segregation of these processes, rather than completely independent coding (Calder and Young 2005).”

“Future research might investigate whether invariant facial features, such as identity information or the color of the skin (e.g., structural features; Kaufmann and Schweinberger, 2004; Calder and Young, 2005; Guerra et al., 2012; Golkar and Olsson, 2017), are the more persistent threat or safety cues in person percept…”

“It is well established that we use facial expressions as a source of nonverbal communication (e.g., Calder & Young, 2005).”

“While there might be overlap between the neural substrates mediating identity and expression analysis (Calder & Young, 2005; Ganel, Valyear, Goshen-Gottstein, & Goodale, 2005), there also appears to be a degree of independence.”

“1 B ), have been defined in the most recent neurofunctional model of human face recognition (Duchaine and Yovel, 2015; for earlier models: Haxby et al., 2000; Calder and Young, 2005; Haxby and Gobbini, 2011).”

“In humans, face recognition is subtended by an extended cortical network of face-selective regions, divided into a ventral and a relatively more dorsal component (Calder & Young, 2005; Duchaine & Yovel, 2015; Haxby, Hoffman, & Gobbini, 2000).”

“Despite extensive research on the recognition of facial identity and facial expression of emotions (see Leeland, 2008 , for an overview), on coding of facial behaviours (Ekman & Rosenberg, 1997 ), and their neuronal foundations (eg, Calder & Young, 2005 ; Haxby, Hoffman, & Gobbini, 2000 ), important questions about fa…”

“These models are largely based on neuropsychological evidence in normal and brain injured patients, electrophysiological studies in non-human primates and functional imaging studies, although evidence is not as strong as is often assumed (for review, see Calder and Young, 2005).”

“A number of studies have shown that the processing of every facial expression elicits activity in face-selective regions of the inferior occipital gyrus (IOG), lateral fusiform gyrus (LatFG) and posterior superior temporal sulcus (pSTS) (Calder and Young, 2005; Haxby et al., 2000).”

“Our study is in agreement with the conclusion that the separate route theory is less well supported (e.g., Calder and Young, 2005; D’Argembeau and Van der Linden, 2007, 2011).”

“Furthermore, evidence suggests that emotion cues are processed prior to facial identity, see review (Calder and Young, 2005), therefore oxytocin may bias processing of emotional cues, which could have subsequently impacted negatively on processing of identity.”

“Future research should assess more directly the neural mechanisms mediating this type of interactive face processing (cf. Calder & Young, 2005; Haxby et al., 2000).”

“When Calder and Young (2005) approached the question more than a decade ago in reviewing studies of the relation between recognition of facial identity and facial expression, the underlying principles and extent of any division of labour remained uncertain.”

“Our hypothetical model space was motivated by a dominant theory (Haxby et al. 2000; Calder and Young 2005), which asserts that ventral connections between the occipital face area (OFA) and the fusiform face area (FFA) are specialized for perceiving identity, while dorsal connections between the OFA…”

“Another structure whose activity increases when decoding emotional facial information is the amygdala (see, for example, Haxby et al., 2000; Calder and Young, 2005; Phelps and LeDoux, 2005; Adolphs, 2008).”

“Most functional models of face perception, such as the IAC model, fail to make a distinction between how faces, voices, and names relate to each other in memory (e.g., Burton et al., 1990; Bruce & Young, 1986; Calder & Young, 2005).”

“…in the perceptual analysis of facial expressions, and this is borne out by a number of studies that have implicated STS in neural responses to expression (Calder & Young, 2005; Psalta, Young, Thompson & Andrews, 2014) and social perception from visual cues (Allison, Puce & McCarthy, 2000).”

“[102]) showed that identity and emotion recognition are both accompanied by an enhanced activity of the Fusiform Face Area as well (for a review of unitary face perception models see [103]), but traditional face perception models assume different neural pathways of identity and emotional face processing [45,46,104].”

“Inroads have been made in the study of the cognitive and neural architecture underlying individual recognition (e.g., Calder & Young, 2005 ), but the appropriate psychophysical tools have only rarely been applied to kin recognition (e.g., Dal Martello & Maloney, 2006 ).”

“…by a body of research that points to its special status within human visual processing (e.g., Ellis, Bruce, & De Schonen, 1992; Kanwisher, McDermott, & Chun, 1997; Tsao & Livingstone, 2008; Vuilleumier & Pourtois, 2007; see Calder & Young, 2005, for a recent review of face processing research).”

“This is consistent with general accounts of recognition memory [Shannon and Buckner, 2004; Wheeler and Buckner, 2003], as well as specific studies of face recognition [Calder and Young, 2005; Ishai and Yago, 2006].”

“…(Bonda et al. 1996; Allison et al. 2000), including bodies, faces, expressions, and mouth and eye movements (Perrett et al. 1992; Bonda et al. 1996; Phillips et al. 1997; Puce et al. 1998; Haxby et al. 2000; Hoffman and Haxby 2000; Beauchamp et al. 2003; Hooker et al. 2003; Calder and Young 2005).”

“…sulcus, a region known for being involved in facial expression recognition and empathy (see Narumoto, Okada, Sadato, Fukui, & Yonekura, 2001; Calder & Young, 2005, for a review), increased significantly when participants watched yawning videos, but not mouth opening videos, and was…”

“At a neural level, although there is some degree of separation between the mechanisms for recognition of identity and expression, these complex tasks involve the concerted action of potentially overlapping functional components [20], which could explain part of this result.”

“The cortical and subcortical brain areas for emotion recognition have been defined, and include amygdala, hippocampus, insula, anterior cingulated cortex, ventral striatum, fusiform gyrus, and orbitofrontal cortex [4, 7, 30] as well as other areas which may play a minor role.”

“…expression are probably not processed completely independently (e.g., Ganel & Goshen-Gottstein, 2004; Schweinberger, Burton, & Kelly, 1999), there is nevertheless some separation between the coding of these two types of facial information (Calder & Young, 2005; Ganel & Goshen-Gottstein, 2004).”

“…might implement mechanisms for generating impressions of states of seeing or visual attention, and Blakemore 5 For further discussion of the nature of these cues see Etcoff and Magee (1992), Calder et al (1996), Calder et al (2000), Calder et al (2001), Adolphs (2002), and Calder and Young (2005).”

“Whereas facial expression categorization occurs in a separate system that includes the occipital cortex, the right posterior superior temporal sulcus (pSTS), and the amygdala (for a review see, Calder and Young, 2005; Pessoa, 2008).”

“Calder and Young (2005) proposed a principal component analysis (PCA) framework for face perception, accounting for such interference effects by way of a single multidimensional face-coding system.”

“Moreover, principal components analyses of facial variations point to distinct overlapping sources of variation between different identities or expressions, suggesting that at the visual level, expression, and generic shape are somewhat confounded (Calder and Young, 2005).”

“In accordance with neuro-computational models of human face processing (Dailey et al., 2002; Calder and Young, 2005), FR detects facial configurations in the following steps (Van Kuilenburg et al.”

“…Sugase et al., 1999), brain-damage-based identity processing difficulties (prosopagnosia) frequently co-occur with expression processing difficulties (Calder and Young, 2005), and expression aftereffects are partially tuned to facial identity, suggesting overlapping neural processing (Campbell and…”

“…have assumed that expression is processed by specialized systems that can be dissociated from those processing fixed properties of the faces, such as gender and identity, though more recent proposals have argued against a strict dissociation between face processing systems (Calder & Young, 2005).”

“…distinction of prosopagnosia and with a sharp functional dissociation as advocated in neuro-cognitive models of face processing between a “structural encoding stage” and a store of “facial recognition units” (Bruce & Young, 1986; Calder & Young, 2005; Duchaine & Yovel, 2015; Haxby et al., 2000).”

“A complex cortical network of brain regions is involved in face recognition (Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000), which includes the fusiform face area (FFA) (Gobbini & Haxby, 2007; Grill-Spector, Knouf, & Kanwisher, 2004; Kanwisher, Mcdermott, & Chun, 1997; Kanwisher & Yovel,…”

“…are different types of facial cues (e.g., changeable vs invariant information) and involve different neural circuits for information processing (Calder & Young, 2005), the recognition performance of these facial cues are subject to the influence from similar gaze behaviour and personal traits,…”

“One way to simultaneously code for multiple types of facial information is through the population coding of broadly tuned neurons (Calder and Young, 2005; Meyers et al., 2015).”

“There are evidences that suggest anatomical separation of neural pathway in facial expression and facial identity recognition in human [12 − 14] .”

“…by the knowledge that PCA relates reliably to human ratings and performance (O’Toole et al., 1994, 1998) and has been proposed as a statistical Frontiers in Psychology | www.frontiersin.org 9 March 2015 | Volume 6 | Article 346 analog of the human representation of faces (Calder and Young, 2005).”

“Calder et al. (2000) showed that it tookmore time to identify the facial expressionononepart of the composite facewhen theotherpart showed a different expression than when they were the same.”

“These multidimensional models provide a powerful representational framework that can account for variations in face identity and facial expressions (Calder & Young, 2005; Neth & Martinez, 2009), race (Caldara & Abdi, 2006; Furl, Jonathon, & O’Toole, 2002; O’Toole, Abdi, Deffenbacher, & Valentin,…”

“…are that processing of faces is dominant over other object classes and that different kinds of facial information like identity, expression and direction of gaze are processed in separate, relatively independent subsystems (e.g., Bruce and Young, 1986; Haxby et al., 2000; Calder and Young, 2005).”

“Emotion recognition activates different brain areas (Bruce & Young, 1986; Calder & Young, 2005) and involves different neural networks (de Gelder, Vroomen, Pourtois, & Weiskrantz, 1999; Morris, Ohman, & Dolan, 1999; Haxby, Hoffman, & Gobbini, 2000).”

“interaction was significant in the amplitude analysis, Task [F (1, 15)=0.”

“Hyper-arousal of the amygdala is clinically important, given its role in appraising the affective salience of stimuli, especially facial expressions (Calder and Young, 2005), and in the appraisal of perceived threat and mediation of fear responses (LeDoux, 1993).”

“…Young model of independent mechanisms for processing of face features, several criticisms of the neuropsychological results had been identified (see Calder and Young, 2005) and subsequent behavioral and neurophysiological studies have found counter evidence, supporting a model of interdependent…”

“…in perceiving both facial identities and expressions (Gorno-Tempini et al., 2001; Vuilleumier et al., 2001; Kaufmann and Schweinberger, 2004; Winston et al., 2004; Calder and Young, 2005; Ganel et al., 2005; Ishai et al., 2005; Fox and Barton, 2007; Palermo and Rhodes, 2007; Fox et al., 2008).”

“This is consistent with the literature that supports an interactive rather than dual route hypothesis of identity and expression processing (e.g., Ganel and Goshen-Gottstein, 2004; Calder and Young, 2005).”

“Whereas identity and emotion processing rely on two dissociated systems in vision and audition (Belin et al., 2004; Bruce & Young, 1986; Calder & Young, 2005; Haxby et al., 2000) and facial familiarity is independent of facial emotion perception (Bruce, 1986; Calder et al., 2000; Campbell et al.,…”

“…for the processing of the facial emotional expressions (Bruce & Young, 1986), more recent findings suggest that the processing of these two types of facial information are not completely independent (Baudouin, Gilibert, Sansone, & Tiberghien, 2000; Calder & Young, 2005; Posamentier & Abdi, 2003).”

“The interpretation of facial expressions is particularly relevant in social anxiety since the human face is central to social cognition and communicates a vast amount of information (Calder & Young, 2005).”

“We thus suggest that the connectivity results tend to be in line with other views that emotion exerts parallel effects on perception (Calder and Young, 2005), attention and memory (Talmi et al., 2008, 2013).”

“Other functional models of face perception have been proposed to account for the hierarchy of processing stages involved in identifying a person from their face (e.g. Bruce & Young, 1986; Burton, Bruce, & Johnston, 1990; Calder & Young, 2005).”

“Although classical models of face recognition have emphasized independent neural pathways for the processing of face identity and face emotion (Bruce and Young, 1986; Haxby et al., 2000), increasing evidence challenges this dichotomy (Calder and Young, 2005; Tsuchiya et al., 2008).”

“This emotional n-back version allows us to investigate different aspects of SWM content updating such as identity and emotional expression [45, 46].”