Understanding the recognition of facial identity and facial expression

“Is multimodal analysis involved? The co-occurrence of impairments in the recognition of facial and vocal expression is not restricted to cases that show disproportionate impairments for one emotion; in fact, the relationship is if anything more striking in patients who have general problems in recognizing facial expre…”

“Besides the FFA's wellestablished role in face identification, which is assumed to rely on mainly invariant facial features (Calder & Young, 2005; Dekowska, Kuniecki, & Ja skowski, 2008; Haxby, Hoffman, & Gobbini, 2002; Hoffman & Haxby, 2000), recent studies highlight the importance of the FFA for…”

“Face perception research has tried to explore the different processing stages in face recognition and their interactions systematically (e.g., Bate & Bennetts, 2015; Calder & Young, 2005).”

“…independency of identification and expression recognition, more recent studies with adults have repeatedly and consistently demonstrated their interaction (Campbell & Burke, 2009; Fox, Moon, Iaria, & Barton, 2009; Leleu et al., 2012; Schweinberger et al., 1999; for review, Calder & Young, 2005).”

“, 1] emotional state [2-5], race [6-9], and gender [10, 11].”

“While it remains to be established how common dissociations between behavioral and neurophysiological deficits are face processing impairments, this topic might provide further support to the relationship between impaired identity and expression recognition [10].”

“In sum, some points regarding the hypothesis that emotion processing and identity processing are independent (for discussion see Calder & Young, 2005; Schweinberger & Burton, 2011; Young & Bruce, 2011) remain to be clarified.”

“…approach detecting specific features such as the corners of the mouth or eyebrow shape (Calder, Burton, Miller, Young, & Akamatsu, 2001; Calder & Young, 2005; Chen et al., 2005; Oliver, Pentland, & Berard, 2000) or a regionbased approach in which facial motions are measured in certain…”

“We used a robotized pan-tilt ASL Model 504 [13] eye tracker, that can keep the eye in its camera’s field of view.”

“According to numerous lesion, neuroimaging, and primate studies, eyes and gaze are processed along a largely posterior network involving the occipital lobe, superior temporal sulcus, fusiform gyrus, intraparietal sulcus, orbitofrontal cortex, and amygdala (Calder and Young 2005).”

“These findings collectively suggest that the face-related N170 and FFA should be better understood as visuoperceptual representations of both facial expression and identity (Calder and Young 2005; Hinojosa et al. 2015).”

“Recognition rates in this study varied considerably across emotion, which were similar to those found in previous research and were likely due to differences in emotion recognition across expressions (Calder & Young, 2016).”

“Facial recognition is a critical component of social cognition, involving a complex interplay of cognitive processes that enable individuals to identify, interpret, and respond to the emotional expressions and identities of others (Calder & Young, 2005).”

“These fi ndings are largely consistent with models of face processing which posit separate visual pathways for the encoding of identity and the encoding of dynamic aspects of face perception including facial expression recognition (e.g. Calder & Young, 2005; Haxby et al., 2000).”

“However, more recent reflections indicate, that such model might be an oversimplification and facial expressive cues may interfere with certain aspects of facial identity, such as person’s sex or age (Calder & Young, 2005).”

“During this period, L1 loss is used to calculate the similarity of the image to avoid excessive blurring of the image to optimize the generator network [5].”

“Faces provide a wealth of information about people including their identity (Ellis, 1975), social traits (Oosterhof and Todorov, 2008), and emotional state (Bruce and Young, 1986; Calder and Young, 2005).”

“However, recent evidence suggested that the perception of facial expression is significantly affected by the identity of the face [5, 6].”

“The smiling mouth of the happy expression has also shown in other studies to be a distinctive and unique facial feature on the happy face, resulting in quicker and more accurate performance (Leppänen et al., 2004; Calder and Young, 2005).”

“La percepción facial implica dos capacidades: la identificación del rostro y el reconocimiento de las expresiones faciales (Calder & Young, 2005), Haxby y Gobbini (2011) proponen un sistema central para el análisis visual de las caras, el cual estaría compuesto por células del área occipital para…”

“The problem of the automated facial expressions recognition has been intensively studied in the literature [5, 19].”

“Functional and neuroanatomical models of face cognition (Bruce & Young, 1986; Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2002) assume two relatively independent processing streams for the recognition of identity and the recognition of facial expressions such as manifestations of emotions.”

“Other neural regions including the superior temporal sulcus, purportedly involved in processing changeable face features, have also been implicated (for reviews see Calder and Young (2005); Haxby et al. (2000)).”

“What is particularly important for our purposes is that recognition of the emotion from the face can be double dissociated from recognition of other information from the face, such as recognizing the identity of a familiar person (Adolphs et al., 1994; Calder & Young, 2005; Tranel et al., 1988).”

“However, it may be that one task is simply easier than the other, thus making it difficult to draw definitive conclusions about the relationship between two processes from single dissociations (Calder & Young, 2005).”

“There is an ongoing debate on whether this component is affected by emotion, since according to traditional models of face perception emotion is encoded in parallel and independently from structural characteristics (Bruce & Young, 1986; Calder & Young, 2005).”

“At the level of the brain, processing of face stimuli is thought to be mediated by neural systems that are distributed across the extrastriate visual areas in the occipitotemporal cortex (Haxby et al., 2000; Calder & Young, 2005).”

“There is strong evidence that facial identity and expression might be processed by separate systems in the brain, or at best they are loosely linked, [ 7 ].”

“Bruce and Young's model ( 1986 ) postulated that facial identification and expression identification processing are independent but emerging research evidence contradicts this (Calder & Young, 2005 ).”

“However, recent image-based analysis and behavioral and functional imaging experiments have shown that facial identity and expression are not necessarily processed by independent perceptual systems; in fact, these systems may be interdependent (Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000).”

“Moreover, Calder and Young (2005) suggested than the recognition of facial identity and facial expression are somewhat related and do not follow strictly separate functional and neural pathways.”

“Facial Expression Recognition (FER) conveys rich information and is a key focus of current AI research [3, 8, 14, 20, 31, 33].”

“Facial expression understanding is a fundamental task in affective computing and plays an important role in many human-centered applications, including human-computer interaction, mental health analysis, and social robotics [1]–[4].”

“Physiological bio-metrics provide stable identity cues [33–36], but often raise privacy concerns.”

“These distinctions are critical because different tasks engage partially distinct components of the broader face and social-cognitive network 11,36 .”

“Face perception is critical for social interaction, enabling the recognition of individual identity and the interpretation of emotional states (Bruce & Young, 1986; Calder & Young, 2005; Ekman & Friesen, 1971).”

“In addition, well-timed facial expressions do not simply mirror the speaker’s stance or display understanding of the speaker’s talk, but also perform systematic operations on the projected course of the talk (Calder and Young, 2016).”

“…the face has a degree of robustness even though visual inputs are always changing (Whitney & Yamanashi Leib, 2018), and the encoding of traits is consistent across individuals, which has led researchers to refer to this as invariance in face processing (Calder & Young, 2005; Todorov et al., 2015).”

“…conversations co-occur or in difficult noise situations, which can providing clarity and focus on non-verbal aspects of communication, such as facial expressions [1, 12, 21, 36], gaze estimation [14, 31, 67, 68, 75], head pose estimation [19, 45, 46, 50, 53], and body language [6, 43, 48, 51].”

“Furthermore, if perceptual experience sometimes presents expression or affect, gradations are permitted: faces can appear more or less angry (Calder & Young, 2005 ).”

“7,48 This could be, at least in part, because tests of the distributed face processing model have often compared the recognition of facial expressions to the recognition of facial identities, and identity judgements require the face stimuli to be familiar to the participants.”

“Based on this evidence, a functional connection between the pSTS and amygdala has been proposed in face processing models (Calder & Young, 2005; Haxby et al., 2000).”

“Faces convey a wide range of information essential for social communication, including identity (Calder & Young, 2005; White & Burton, 2022), gaze (Dalmaso et al., 2020; Frischen et al., 2007; Kleinke, 1986), emotion (Calder & Young, 2005; Calvo & Nummenmaa, 2016; Wieser & Brosch, 2012), and social…”

“…pathway is related to facial changeable information, such as expression, gaze, and lip movement, and involves the superior temporal sulcus (STS), whereas the other is related to facial changeless information, such as facial identity, and involves the lateral fusiform gyrus (Calder and Young, 2005).”

“…[7] and behavioral studies [8,9], even though some authors have claimed that the clear evidence for the separation between the processing of facial identity and expression might also be explained by different frameworks, other than the independence of the corresponding visual pathways [10].”

“It has been proposed, although not universally accepted, that the mechanisms underlying the perception of identity and emotion are to some degree separable (see [26] (Calder & Young, (2005) for a review).”

“…the observed for external features acting as Prime1, with a relevant participation of multiple regions along the entire right face pathway (Calder and Young, 2005; Duchaine and Yovel, 2015; Gobbini and Haxby, 2007; Guntupalli et al., 2017; Haxby et al., 2000), as well as of left visual…”

“Previous research indicates that people can recognize facial expressions by contrasting expressions with neutral faces from the same subject [4], suggesting that a facial expression can be decomposed into an expressive component and neutral component [5].”

“However, evaluations that are conducted with isolated images or drawings may be limited without having contextual information (Calder et al., 2005).”

“Humans primarily identify others by looking at their faces, so their facial recognition abilities are highly developed [1].”

“Conversely, the converging evidence suggested that the perceived emotional expression of a face is affected by the facial identity of that person, thus, these two aspects of a face are inter-dependent with each other [6,7].”

“…that all facial aspects (e.g., identity and expression recognition) are processed using the same early visual mechanisms (Bruce & Young, 1986; Calder & Young, 2005; Haxby et al., 2000; Pitcher, Walsh, et al., 2011) before diverging at higher levels of processing, rather than as dissociable…”

“These results support the revised model of fusiform function that posits the fusiform contributes to structural encoding of facial expression during the initial stages of processing (Calder and Young, 2005; Duchaine and Yovel, 2015), with the notable addition that it may be primarily posterior fusiform contributes to…”

“The difference between these two methods is that [2] proposed the dissociation immediately after the face reading state (structural encoding) whereas [4] suggested after a common representational system that codes facial identity and expression together.”