Understanding the recognition of facial identity and facial expression

“Is multimodal analysis involved? The co-occurrence of impairments in the recognition of facial and vocal expression is not restricted to cases that show disproportionate impairments for one emotion; in fact, the relationship is if anything more striking in patients who have general problems in recognizing facial expre…”

“Models of face processing propose that the human STS is a core node of a distributed face network (Adolphs, 2002; Calder & Young, 2005; Haxby et al., 2000).”

“In the face-recognition literature, a distinction has been drawn between identity and expression processing (for a review, see Calder & Young, 2005).”

“The ANOVA revealed a significant main effect of electrode cluster [F(1, 16) = 16.”

“Others have challenged this idea that the processing of expression and identity can be fully dissociated in the face network at the cognitive level and at the neural implementation level (e.g., Calder & Young, 2005).”

“Certainly the Bruce and Young model has dominated our understanding of the architecture of face perception and where challenges have been made none have offered a widely accepted alternative (Calder & Young, 2005).”

“In accordance with neuro-computational models human face processing (Dailey et al., 2002; Calder and Young, 2005), FR detects facial configurations in the following steps (Van Kuilenburg et al., 2005, 2008): (1) The C ascade classifier algorithm finds the position of the face (Viola and Jones, 2004).”

“Our findings are thus perhaps most consistent with accounts positing a partial or graded, rather than absolute, segregation of facial identity and expression processing in the brain (Calder and Young, 2005; Dahl et al., 2016), with processing in more anterior brain regions weighted towards ex-”

“…of neuropsychological and neuroimaging evidence in support of separate visual processing streams for facial identity and facial expression, Calder and Young (2005) argue that the cognitive and neural mechanisms involved in the perceptual analysis of facial identity and expression show…”

“It has been suggested that, in face processing, visual speech is processed very distinctly from visual identity and emotional expression information, which are seen to share many properties at perceptual levels (Calder and Young, 2005).”

“More recently, WM tasks have been used to identify separable neuroanatomical networks for changeable (e.g., expression) vs. invariant (e.g., identity) facial features consistent with previous theories and data (e.g., Haxby et al., 2000; Calder and Young, 2005).”

“Changeable cues, such as facial expressions, require monitoring throughout social interactions ( Calder & Young, 2005 ).”

“The principles of FPS-fMRI validated here with a functional face localizer could be applied straightforwardly to refine our knowledge of the functions of these regions with respect to face identity and facial expression processing for instance (Calder and Young 2005; Haxby et al. 2000; Duchaine and Yovel 2015), to map…”

“In this system, the dissociation between codes emerges not from separate neural modules [2], but rather from statistical regularities inherent in the visual input itself, which are processed in parallel [11].”

“Human faces provide a wealth of socially relevant information regarding a person’s gender, age, and race (Jack & Schyns, 2015; Calder & Young, 2005).”

“…is growing evidence for an important role of the human STS in the perception of facial emotional expressions (Adolphs, 2002; Allison et al., 2000; Calder and Young, 2005; Engell and Haxby, 2007; Furl et al., 2007; Haxby et al., 2002; Kret et al., 2011; LaBar et al., 2003; Narumoto et al., 2001;…”

“These results are consistent with the suggestion that the OFA plays a critical early role in the analysis of faces by detecting face features to construct an initial representation of a face image (Calder and Young, 2005; Haxby et al., 2000; Pitcher et al., 2007).”

“…sulcus (pSTS) and the amygdala (for a review see, Calder & Young, 2005; Pessoa, 2008). ever, some authors have questioned the idea of independence between those systems, by mainly relying on results from computational modelling and neuroimaging evidence (Calder, 2011; Calder & Young, 2005 ).”

“…gleaned from faces are processed by the same cognitive mechanism (single route) or by multiple cognitive mechanisms (parallel routes) is an enduring question (Bruce & Young, 1986; Burton, Bruce, & Johnston, 1990; Calder & Young, 2005; Ganel & Goshen-Gottstein, 2002; Goshen-Gottstein & Ganel, 2000).”

“The processing of facial expressions has been more widely studied than prosody (see [46–48] for reviews).”

“While it has been argued that recognizing a face is a separable process from recognizing facial expressions (Calder & Young, 2005 ), other studies have shown that emotional information can affect the identification of faces and other stimuli (Buchanan & Adolphs, 2003 ; Gupta & Srinivasan, 2009 ).”

“Previous studies suggest that the fusiform gyrus plays a crucial role in facial recognition, especially invariant aspects of faces.(32,33) Compared with the healthy control group, the activation in the fusiform gyrus indicates the failure of the Internet gaming group to ignore the emotional interfering angry faces dur…”

“The finding that emotional facial expressions lead to enhanced pain ratings independent of their valence may be linked to the important role of facial expression processing in the human brain, which involves a widespread network of cortical and subcortical structures (Adolphs, 2002; Calder and Young, 2005).”

“On the other hand, the notion that facial expression information is coded completely free from identity-based visual information has been met with some criticism (Calder & Young, 2005; Ganel & Goshen-Gottstein, 2004).”

“…may be adding noise to the literature is that studies where face kinematics have been manipulated often employ stimuli where features such as identity, gender, age and other observable demographic information can provide additional cues about the underlying emotional state (Calder & Young, 2005).”

“The recognition of facial expressions and their direction of gaze is a prerequisite for social behavior of both human and nonhuman primates (Adolphs, 2001; Haxby et al., 2002; Calder and Young, 2005; Hadjikhani et al., 2008) and a large prior literature has implicated the amygdala as a key component into the processin…”

“…such as judgement of age (Ellis & Young, 1988) ; or alternatively facial expression and facial identity information is extracted differentially from a common representational system, resulting in a relative rather than an absolute dissociation between these two facial cues (Calder & Young, 2005).”

“This is remarkable in that face detection is a prerequisite to any further processing, such as identification or expression analysis, which are tasks that have received a great deal of attention by psychologists (see e.g., Burton, Jenkins, Hancock, & White, 2005; Calder & Young, 2005).”

“Our results, thus, seem more consistent with recent proposals that have questioned the validity of the traditional approach to face recognition, in terms of independent visual pathways for processing different properties of faces (see, e.g., Calder & Young, 2005; Ganel & Goshen-Gottstein, 2002).”

“The N170 is Sensitive to Long-term (Personal) Familia models of human face recognition (Bruce and Young, 1986; Calder and Young, 2005; Young and Bruce, 2011).”

“…among the experience of anger, disgust, and rejection in AN might generalise to the process of recognising these emotions in others, as there is evidence that the same brain regions implicated in experiencing emotions are involved in interpreting emotion from facial signals (Calder & Young, 2005).”

“Some research have shown that human can recognize facial expression through the dynamics of facial proression from the neutral face to the expressive face [7], [8].”

“…convey a lot of information not only on identity and other stable aspects of a person, but also on the emotional state and attentional focus, and therefore can be regarded as highly important for everyday interactions (Calder and Young, 2005; Leopold and Rhodes, 2010; Tsao and Livingstone, 2008).”

“…specified distinct face and body latent factors but allowed an additional supramodal latent factor, in line with research demonstrating that processes underlying emotion recognition are closely integrated across communicative modalities (Calder & Young, 2005; de Gelder, 2006; Peelen et al., 2010).”

“…some evidence is available that prosopagnosic participants have similar difficulties in processing emotional expressions (Humphreys et al., 2006, see also Calder and Young, 2005 for a review), although not all prosopagnosics show impaired emotional expression processing (Le Grand et al., 2006).”

“Such findings are supported by numerous studies of patients reporting disturbances of facial expression analysis occurring alongside the spared ability to accurately recognise familiar faces (Hornak, Rolls, & Wade, 1996 ; Parry et al, 1991 ), and vice versa (Bruyer et al, 1983 ; Tranel, Damasio, & Damasio, 1988 – but…”

“Instead, evidence was more consistent with a psychological constructionist approach (Barrett and Bliss-Moreau, 2009; Calder and Young, 2005; Haxby et al., 2000), where a network of interacting regions involved in both emotional and non-emotional stimuli, are activated during facial emotion…”

“…the featural dimensions of human face representation, can be defined using principal component analysis (PCA) on face images or using parameterized models that are generative for constructing what appear to be realistic new face stimuli (Calder & Young, 2005; Freiwald, Tsao, & Livingstone, 2009).”

“…are a specific prediction of recent models of face processing that have emphasised interactions between different cues when processing faces (eg Calder and Young 2005; Haxby et al 2000), and problematic for alternative models in which functionally distinct facial characteristics are thought to…”

“However, others have challenged the idea of independent pathways, noting that dissociations between deficits need not arise at a perceptual level and that the selectivity of neurons and neural areas for these attributes is far from complete (for reviews see Calder, 2011; Calder & Young, 2005).”

“Moreover, even human subjects fare better at face recognition than voice recognition in many situations (Colavita, 1974; Hanley et al., 1998; Joassin et al., 2004; Calder and Young, 2005; Sinnett et al., 2007).”

“Calder and Young (2005) share a similar view.”

“, 2014), other studies provide evidence that these processes are at least partly overlapping (Calder and Young, 2005; Fisher et al., 2016).”

“More recent advances have come from mappings from cognitive/functional descriptions of the face-processing architecture onto neural systems which implement these functions (Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000).”

“However, less is known about the neural mechanisms that the STS and associated visual areas employ to derive expression percepts from face dynamics (Calder and Young 2005).”

“The theoretical basis of these models is largely derived from single-cell recordings in macaques, lesion studies and patient-based research; however, experimental evidence for these models has shown inconsistencies (for review, see Calder and Young 2005 ).”

“Consistent activations were found in the lateral fusiform gyrus centrally involved in facial expression recognition (Calder and Young 2005), as well as posterior polysensory areas of the superior temporal sulci and gyri that integrate multitude of social signals and act as the centralized “hub” of…”

“, 1999), there is no clear evolutionary bases for a left hemispheric specificity for processing affective facial expressions (Calder and Young, 2005).”

“The view that pSTS is exclusively involved in processing dynamic aspects of faces is not strongly supported by the empirical evidence (Calder and Young 2005).”

“For example, to resolve the different tasks of identity and expression categorization (using the same stimulus), different face information is needed (e.g., Morrison and Schyns, 2001; Schyns et al., 2002).”

“These regions have since been promoted as “core nodes” in an extended face processing network (e.g., Haxby et al., 2000; Rossion et al., 2003; Calder and Young, 2005; Ishai, 2008; Pitcher et al., 2011).”

“Nevertheless, most of the neuroimaging studies published so far were focused on the functional selective role of each region separately (Calder and Young, 2005).”

“To recognise, classify, and categorize the objects contained in the images, all of these detection approaches employ CNN methods like GoogleNet [17], AlexNet [15], and VGG16 [15], among others [18].”

“Given these difficulties, it is not surprising that special brain mechanisms have developed for the identification of familiar faces (Haxby et al., 2000; Calder and Young, 2005; Gobbini and Haxby, 2006, 2007; Duchaine and Yovel, 2015; Ramon and Gobbini, 2018; Young and Burton, 2018; Rapcsak, 2019).”

“Much has been learned about face recognition and the perception of changeable facial qualities such as eye gaze, smiling, and other emotional expressions (Bruce & Young, 1986; Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2002).”

“The concordance between our expression and identity findings hints at a common substrate where identity and expression form part of a common distributed representation (Calder et al. 2001; Calder & Young 2005).”

“Published by Elsevier Inc a b s t r a c t a r t i c l e i n f o”

“Given the importance of motion for facial expressions, we focused on this aspect of face perception but it should be noted that identity and expressions may be processed by different pathways in the brain (Bruce & Young, 1986; Haxby, Hoffman, & Gobbini, 2002; but see also Calder & Young, 2005).”