The DOI matches its Crossref record; 107 of the 111 references it cites were found in a registry or an index; none of the 124 sources its claims cite is checked for support yet.
Automatic analysis of facial expressions: the state of the art
Open accessAbstract
Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. There are several related problems: detection of an image segment as a face, extraction of the facial expression information, and classification of the expression (e.g., in emotion categories). A system that performs these operations accurately and in real time would form a big step in achieving a human-like interaction between man and machine. The paper surveys the past work in solving these problems. The capability of the human visual system with respect to these problems is discussed, too. It is meant to serve as an ultimate goal and a guide for determining recommendations for development of an automatic facial expression analyzer.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This survey reviews the state of the art (through 2000) in automatic facial expression analysis, decomposing the problem into face detection, facial expression data extraction, and facial expression classification. It compares surveyed systems against an "ideal analyzer" modeled on the human visual system and concludes that none of the three subproblems has been solved in the general case.
Key findings
- The paper identifies three related subproblems — face detection, facial expression information extraction, and facial expression classification — and argues an ideal system must automate all three, ideally in real time for HCI use.
- Early systems (1991–1995) achieved correct recognition rates of 70–92 percent when recognizing 3–8 emotion categories; none performed quantified expression classification in terms of facial actions, and only Moses et al. [57] performed in real time.
- Among the surveyed recent systems, reported recognition rates include 92 percent (Yoneyama et al., four expressions), 89 percent for familiar subjects and 73 percent for unknown persons (Hong et al.), 84.5 percent (Huang and Huang), 92 percent generalization / 75 percent for a novel subject (Lyons et al.), 90.1 percent (Zhang et al.), 92 percent upper-face AUs and 86 percent lower-face AUs (Pantic and Rothkrantz), 92/88/83 percent for eyebrow/eye/nose-mouth regions (Cohn et al.), 98 percent (Essa and Pentland), 95 percent (Wang et al.), and 88 percent (Black and Yacoob).
- Most systems rely on frontal-view images of faces without facial hair or glasses, assume constant illumination and immobile subjects, and perform only singular classification into Ekman's six basic emotion categories; only two systems ([62], [106]) perform quantified classification into multiple emotion categories, and none quantifies facial action codes or distinguishes all 44 FACS AUs.
Methods
- Design: a survey/comparative review of vision-based automatic facial expression analysis, covering work from the mid-1970s onward with emphasis on the late 1990s; systems are tabulated (Tables 2–8) against properties of an "ideal analyzer" (Table 1).
- Scope: selectively discusses systems addressing face detection, facial expression data extraction (template-based vs. feature-based; holistic, analytic, and hybrid face representations), and classification (template-, neural-network-, and rule-based methods) from both static images and image sequences.
- Analysis: qualitative comparison of reported techniques, assumptions, and recognition rates; the authors deliberately refrain from ranking systems, arguing that no single common testing database exists.
Limitations
- Stated by the paper: no single well-defined, commonly used testing database exists, so the authors deliberately do not rank systems objectively; several surveyed methods were tested only on the images used for training, so their person-independence is uncertain; most systems assume frontal views, constant illumination, no facial hair or glasses, and immobile subjects, which is unrealistic for the target application domains; only two systems handle automatic face detection in an arbitrary scene and only three deal to some extent with rigid head motions; only Essa and Pentland's method handles faces with facial hair and/or eyeglasses; no system performs quantification of facial action codes or distinguishes all 44 FACS AUs.
- Evident from the text: the survey is selective rather than exhaustive, so coverage of the field is incomplete by the authors' own statement; the "ideal analyzer" is a normative construct rather than an implemented benchmark, and the comparison is largely qualitative with heterogeneous, non-comparable evaluation protocols across systems.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
95 Not checked yet · 42 No basis given · 19 Own interpretation · 3 Own result · 1 Own method
In the paper
“So far, the majority of studies treat various human communication channels separately, as indicated by Nakatsu [58].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 1 INTRODUCTION
What the sources say
- Not checked yetcites Nakatsu [58]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Relatively few existing works combine different modalities into a single system for human communicative reaction analysis.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 1 INTRODUCTION
What the sources say
- Not checked yetcites [9] source
- Not checked yetcites [15]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Most of the studies on automated expression analysis perform an emotional classification.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 3, several surveyed systems can be classified as methods for facial expression analysis from static images.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“None of these systems performs a quantified expression classification in terms of facial actions.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The correct recognition rates reported range from 70-92 percent when recognizing 3-8 emotion categories.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This issue has not been handled by the currently existing systems.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“In general, three types of face representation are mainly used in facial expression analysis: holistic (e.g., isodensity maps [38]), analytic (e.g., deformable templates [105]), and hybrid (e.g., analytic-to-holistic approach [45]).”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2 Facial Expression Data Extraction
What the sources say
- Not checked yetcites [38]
- Not checked yetcites [105] source
- Not checked yetcites [45]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Also, except the system proposed by Moses et al. [57], no system listed in Table 2 performs in real-time.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
What the sources say
- Not checked yetcites [57] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nevertheless, most of the studies on vision-based facial expression analysis rely on Ekman's emotional categorization of facial expressions.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 58
- It may state two separate claims as one. 47
- The quote only partly states this claim. 6
In the paper
“Although Hong et al. utilize the PersonSpotter system [81], which deals with real-time processing of video sequences, they perform facial expression analysis from static images.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [81] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“No facial hair or glasses are allowed.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.1 Face Detection in Facial Images
What the sources say
- Not checked yetcites [62] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The observed subject should face the camera while siting at approximately 1m distance in front of it.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.1 Face Detection in Facial Images
What the sources say
- Not checked yetcites [42]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“None of these systems are able to perform facial expression information extraction from images of occluded faces or a quantified expression classification in terms of facial actions.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These systems primarily concern facial expression animation and do not attempt to classify the observed facial expression either in terms of facial actions or in terms of emotion categories.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The AAM search algorithm has been tested on 100 hand labeled face images.Of these, 19.2 percent failed to converge to a satisfactory result [12].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [12] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Since the mid 1970s, different approaches are proposed for facial expression analysis from either static facial images or image sequences.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“The method of Pentland et al. [65] is real-time and has been successfully tested on a database of 7,562 images of some 3,000 people of both sexes, ranged in age and ethnicity, having varying head positions, headwear, and facial hair.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.2 Face Detection in Arbitrary Images
What the sources say
- Not checked yetcites [65]
plainn = 3,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Depending on the face model a template-based or a feature-based method is applied for facial expression data extraction.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2 Facial Expression Data Extraction
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The utilized techniques include holistic spatial analysis [13], [70], [54], spatio-temporal analysis [52], [100], [73], [57], and analytic spatial analysis [93], [43], [44], [92], [40].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The method works with images of faces without facial hair and glasses, which are hand-labeled with the landmark points beforehand approximated with the proposed AAM.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [18] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Some of the methods listed in Table 2 do not perform automatic facial data extraction (see column 8); the others achieve this by using facial motion analysis [52], [100], [101], [73], [57].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
What the sources say
- Not checked yetcites [52]
- Not checked yetcites [100]
- Not checked yetcites [101] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Independently of the kind of input imagesÐfacial images or arbitrary imagesÐdetection of the exact face position in an observed image or image sequence has been approached in two ways.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1 Face Detection
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Two of the works surveyed in this paper perform automatic face detection in an arbitrary scene.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.2 Face Detection in Arbitrary Images
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The face should be without facial hair and glasses, no rigid head motion may be encountered and illumination variations must be linear for the system to work correctly.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.1 Face Detection in Facial Images
What the sources say
- Not checked yetcites [32]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Steffens et al. [81] reported that their system performs well in the presence of background motion, but fails in the case of covered or too much rotated faces.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.2 Face Detection in Arbitrary Images
What the sources say
- Not checked yetcites [81] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“It is not stated which method has been applied and no limitation of the used method has been reported by Yoneyama et al.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.1 Face Detection in Facial Images
What the sources say
- Not checked yetcites [104]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Still, there is no psychological scrutiny on this topic.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The camera is either mounted on a helmetlike device worn by the subject (e.g., [62], [59]) or placed in such a way that the image has the face in frontal view.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1 Face Detection
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A first category of these utilizes a holistic or a hybrid approach to face representation (Table 5) and applies a template-based method for facial expression information extraction from an input image.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The face should be without facial hair and glasses and in a direct face-to-face position with the camera [99].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1.1 Face Detection in Facial Images
What the sources say
- Not checked yetcites [99] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The surveyed facial expression analyzers classify the encountered expression (i.e., the extracted facial expression information) either as a particular facial action or a particular basic emotion.Some of the systems perform both.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The system cannot deal with minor inaccuracies of the extracted facial data and it deals merely with images of faces without facial hair or glasses.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.2 Facial Data Extraction from Static Images: Feature-Based Methods
What the sources say
- Not checked yetcites [62] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A first category of the surveyed methods for automatic expression analysis from static images applies a template-based method for expression classification.The methods in this category perform expression classification into a single basic emotion category.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.1 Expression Classification from Static Images: Template-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In the approach proposed by Black and Yacoob, the initial regions for the head and the facial features were selected by hand and thereafter automatically tracked.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.3 Facial Data Extraction from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [2] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Zhang et al. deal only with PST Â PST pixels frontal view images of nine female Japanese subjects, manually normalized so that the distance between the eyes is 60 pixels.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [106]
plainn = 9- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The method has been applied to frontal-view facial image sequences.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.3 Facial Data Extraction from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [24] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A similar face representation was recently used by Lyons et al. [51] for expression classification into the six basic plus ªneutralº emotion categories.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [51] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The used PDM has been generated from 90 facial feature points that have been manually localized in 90 images of 15 Chinese subjects showing six basic emotions (Figs. 3 and 4).”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [32]
plainn = 15- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Independent of the used classification categories, the mechanism of classification applied by a particular surveyed expression analyzer is either a template-based-or a neural-network-based-or a rule-based-classification method.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“However, this issue has not been discussed by Hong et al. [30].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [30] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“However, Kimura and Yachida have not discussed this issue.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.3 Facial Data Extraction from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [41]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A first category of the surveyed approaches to automatic facial expression analysis from image sequences uses a holistic or a hybrid approach to face representation (Table 3, Table 5) and applies a template-based method for facial expression information extraction from an input image sequence.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.3 Facial Data Extraction from Image Sequences: Template-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The second category of the surveyed methods for automatic facial expression analysis from static images uses an analytic approach to face representation (Table 3, Table 5) and applies a feature-based method for expression information extraction from an input image.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.2 Facial Data Extraction from Static Images: Feature-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Zhao et al. [107] also utilize a point-based frontal-view face model but do not deal with automatic facial expression data extraction.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.2 Facial Data Extraction from Static Images: Feature-Based Methods
What the sources say
- Not checked yetcites [107] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The face should be without facial hair and glasses and no rigid head motion may be encountered for the method to work correctly.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [104]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Padgett and Cottrell [61] also use a holistic face representation, but they do not deal with facial expression information extraction in an automatic way.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [61] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The initial location of the FFPs in the first frame of an input image sequence is assumed to be known.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.3 Facial Data Extraction from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [95] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The performance of the detection scheme was tested on 496 dual views.Human observers in 89 percent approved when visually inspected the achieved localization of the facial features.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.2 Facial Data Extraction from Static Images: Feature-Based Methods
What the sources say
- Not checked yetcites [62] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A second category of the surveyed methods for automatic facial expression analysis from static images applies a neural network for facial expression classification.Except the method proposed by Zhang et al. [106], the methods belonging to this category perform facial expression classification into a single basic emotion category.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.2 Expression Classification from Static Images: Neural-Network-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The real-time system developed by Kobayashi and Hara works with online taken images of subjects with no facial hair or glasses facing the camera while siting at approximately 1m distance from it.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.2 Facial Data Extraction from Static Images: Feature-Based Methods
What the sources say
- Not checked yetcites [42]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The first category of the surveyed methods for automatic facial expression analysis from facial image sequences applies a template-based method for expression classification.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
plainn = 100- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Also, the method does not deal with blends of emotional expressions.For example, a blend of the angry and scared expressions is recognized as disgust.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.5 Expression Classification from Image Sequences: Rule-Based Methods
What the sources say
- Not checked yetcites [2] source
- Not checked yetcites [3]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Some common assumptions are: the images contain frontal facial view, the illumination is constant, the light source is fixed, the face has no facial hair or glasses, the subjects are young (i.e., without permanent wrinkles), and of the same ethnicity.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 5.1 Detection of the Face and Its Features
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Mehrabian indicated that the verbal part (i.e., spoken words) of a message contributes only for 7 percent to the effect of the message as a whole, the vocal part (e.g., voice intonation) contributes for 38 percent, while facial expression of the speaker contributes for 55 percent to the effect of the spoken message [55].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 1 INTRODUCTION
What the sources say
- Not checked yetcites [55] source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As indicated by Ellis [23], human encoding of the visual stimulus (face and its expression) may be in the form of a primal sketch and may be hardwired.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.2 Facial Expression Data Extraction
What the sources say
- Not checked yetcites Ellis [23]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“In a form of rules, FACS provides a linguistic description of all possible, visually detectable, facial changes in terms of 44 so-called Action Units (AUs).”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
What the sources say
- Not checked yetcites [21]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The terms ªface-to-faceº and ªinterfaceº indicate that the face plays an essential role in interpersonal communication.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 1 INTRODUCTION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“It is generally believed that two-gray-levels images of 100 to 200 pixels form a lower limit for detection of a face by a human observer [75], [8].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.1 Face Detection
What the sources say
- Not checked yetcites [75] source
- Not checked yetcites [8]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Ekman defined six such categories, referred to as the basic emotions: happiness, sadness, surprise, fear, anger, and disgust [19].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
What the sources say
- Not checked yetcites [19] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While the human mechanisms for face detection are very robust, the same is not the case for interpretation of facial expressions.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In practice, it may not be possible to define a face model that can satisfy both, to reflect each and every change in facial appearance and whose features are detectable in a facial image or image sequence.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 3 AN IDEAL SYSTEM FOR FACIAL EXPRESSION ANALYSIS
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“It is interesting to note that the appearance of the upper face features plays a more important role in face interpretation as opposed to lower face features [22].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
What the sources say
- Not checked yetcites [22] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Recently, the problem of automatic face detection in an arbitrary scene has drawn great attention (e.g., see [74], [83], [85]).”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.1 Face Detection
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: In most real-life situations where an automated facial expression analyzer is to be employed, the location of a face in the image is not known a priori.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nevertheless, the information about the context in which a facial expression appears is very difficult to obtain in an automatic way.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The approaches that have been explored lately also include systems for automatic analysis and synthesis of facial expressions [86], [56], [39], [47], [89], [88], [53], [14], [19]”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4 AUTOMATIC FACIAL EXPRESSION ANALYSIS
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“He described each basic emotion in terms of a facial expression that uniquely characterizes that emotion.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
What the sources say
- Not checked yetcites [19] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“First, Ekman's description of the six prototypic facial expressions of emotion is linguistic (and, thus, ambiguous).”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Also, it is not at all certain that each facial expression able to be displayed on the face can be classified under the six basic emotion categories.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is often very difficult to determine the exact nature of the expression on a person's face.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“According to Bassili [1], a trained observer can correctly classify faces showing six basic emotions with an average of 87 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
What the sources say
- Not checked yetcites Bassili [1]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The face should be without facial hair/ glasses, no rigid head motion may be encountered, the first frame should be an expressionless face, and the facial landmark points should be marked in the first frame for the method to work correctly.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.4 Facial Data Extraction from Image Sequences: Feature-Based Methods
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The generalization rate was 75 percent for recognition of expression of a novel subject.”
§ Automatic Analysis of Facial Expressions: The State of the Art > TABLE 8 The Methods for Facial Expression Emotional Classification
What the sources say
- Not checked yetcites [51] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“The time necessary for performing a full analysis of an incoming facial image is about eight seconds.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.1 Expression Classification from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [30] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are infinitely a lot of combinations of different facial actions and their intensities that should be modeled with a finite set of templates.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3 Facial Expression Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is not known how the method will behave in the case of an unknown subject.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.1 Expression Classification from Static Images: Template-Based Methods
plainn = 25- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is not known how the method will behave in the case of unknown subjects.”
§ Automatic Analysis of Facial Expressions: The State of the Art > TABLE 8 The Methods for Facial Expression Emotional Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problem becomes even more difficult due to the fact that everybody has his/her own maximal intensity of displaying a certain facial action.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3 Facial Expression Classification
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The average recognition rate was 85 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.2 Expression Classification from Static Images: Neural-Network-Based Methods
What the sources say
- Not checked yetcites [42]
plainn = 15- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The generalization rate was 92 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > TABLE 8 The Methods for Facial Expression Emotional Classification
What the sources say
- Not checked yetcites [51] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
In the paper
“The dense model-graph seems very suitable for facial action coding based on the extracted deformations of the graph.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.1 Facial Data Extraction from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [30] source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, the descriptions of the emotional expressions, given in terms of facial actions, are incomplete.For example, an expression with lowered mouth corners and raised eyebrows will be classified as sadness.”
§ Automatic Analysis of Facial Expressions: The State of the Art > TABLE 8 The Methods for Facial Expression Emotional Classification
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The achieved recognition rate was 89 percent in the case of the familiar subjects and 73 percent in the case of unknown persons.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.1 Expression Classification from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [30] source
plainn = 25- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The average recognition rate was 92 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > TABLE 8 The Methods for Facial Expression Emotional Classification
What the sources say
- Not checked yetcites [104]
plainn = 10- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The images used for training of the networks were also used for their testing.”
§ Automatic Analysis of Facial Expressions: The State of the Art > TABLE 8 The Methods for Facial Expression Emotional Classification
What the sources say
- Not checked yetcites [104]
plainn = 10- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The method seems suitable for facial action encoding based on the extracted deformations of the net.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.2.3 Facial Data Extraction from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [41]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“The expression recognition performance of the AAM has been trained and tested on P Â PHH images of six basic emotional expressions shown by 25 subjects.The images were chosen for limited pose and lightning variation.The achieved recognition rate for the six basic and ªneutralº emotion categories was 74 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.1 Expression Classification from Static Images: Template-Based Methods
What the sources say
- Not checked yetcites [18] source
plainn = 25- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We are doing so because a typical neural network can perform a quantified facial expression categorization into multiple classes while, in general, the template-based methods cannot achieve such a performance.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3 Facial Expression Classification
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
In the paper
“The average recognition rate was 92 percent for the upper face AUs and 86 percent for the lower face AUs.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.3 Expression Classification from Static Images: Rule-Based Methods
What the sources say
- Not checked yetcites [62] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is not known how the method will behave in the case of an unknown subject.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.1 Expression Classification from Static Images: Template-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Black and Yacoob did not give a full list of the midlevel predicates and the number of different facial actions that the method can recognize is not known.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.5 Expression Classification from Image Sequences: Rule-Based Methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The average process time was 2.5s/frame.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [95] source
plainn = 8- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
In the paper
“The average recognition rate was 95 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [95] source
plainn = 8- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“The dual-views used for testing of the system have been recorded under constant illumination and none of the subjects had a moustache, a beard, or wear glasses.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.3 Expression Classification from Static Images: Rule-Based Methods
What the sources say
- Not checked yetcites [62] source
plainn = 8- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have identified three different but related aspects of the problem: face detection, facial expression information extraction, and facial expression classification.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 6 CONCLUSION
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A very small number of training examples (six sequences) and an insufficient diversity of the subjects (one person) have probably caused this.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
hedgedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Another appealing but still not investigated property of the human visual system is assigning a higher ªpriorityº to the upper face features than to the lower face features since they play a more important role in facial expression interpretation [22].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 5.2 Facial Expression Classification
What the sources say
- Not checked yetcites [22] source
hedgeddirect- Asserted: fails — The authors may not put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“A correct recognition ratio of 91 percent has been reported.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.3 Expression Classification from Static Images: Rule-Based Methods
What the sources say
- Not checked yetcites [62] source
plainn = 8- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“The HMM was trained on 120 image sequences, shown by two male subjects.The method was tested on image sequences shown by the same subjects.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [60] source
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The accuracy of the classification was 92 percent for the eyebrow region, 88 percent for the eye region and 83 percent for the nose/ mouth region.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [10] source
plainn = 100- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The method proposed by Cohn et al. deals neither with image sequences containing several facial actions in a row, nor with inaccurate facial data, nor with facial action intensity (yet the concept of the method makes it possible).”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.4 Expression Classification from Image Sequences: Template-Based Methods
What the sources say
- Not checked yetcites [10] source
plainn = 100- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The NN was trained and tested on the whole set of data (94 images) with 100 percent recognition rate.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.2 Expression Classification from Static Images: Neural-Network-Based Methods
What the sources say
- Not checked yetcites [107] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The achieved recognition rate was 88 percent.”
§ Automatic Analysis of Facial Expressions: The State of the Art > 4.3.5 Expression Classification from Image Sequences: Rule-Based Methods
What the sources say
- Not checked yetcites [2] source
- Not checked yetcites [3]
plainn = 40- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
The paper relies on this without support.
A face is perceived as a whole, not as a collection of the facial features.
“Another characteristic of the human visual system is that a face is perceived as a whole, not as a collection of the facial features.”
The paper relies on this without support.
When a face is partially occluded, humans perceive a whole face as if the perceptual system fills in the missing parts.
“When a face is partially occluded (e.g., by a hand), we perceive a whole face, as if our perceptual system fills in the missing parts.”
The paper relies on this without support.
Each person has his/her own maximal intensity of displaying a particular facial expression.
“On the other hand, each person has his/her own maximal intensity of displaying a particular facial expression.”
The paper relies on this without support.
In most real-life situations, complete immovability of the observed subject cannot be assumed.
“In most real-life situations, complete immovability of the observed subject cannot be assumed.”
The paper relies on this without support.
Considering the state-of-the-art in image processing techniques, inaccurate, noisy, and missing data could be expected.
“Considering the stateof-the-art in image processing techniques, inaccurate, noisy, and missing data could be expected.”
The paper relies on this without support.
Long delays make interaction desynchronized and less efficient.
“Long delays make the interaction desynchronized and less efficient.”
The paper relies on this without support.
In most real-life situations where an automated facial expression analyzer is to be employed, the location of a face in the image is not known a priori.
“Yet, in most of the real-life situations where an automated facial expression analyzer is to be employed (e.g., in a multimodal/media HCI), the location of a face in the image is not known a priori.”
The paper relies on this without support.
In most real-life situations, the observed subject will not remain immovable.
“Also, in most of the real-life situations, it cannot be assumed that the observed subject will remain immovable, as assumed by some methods.”
The paper relies on this without support.
Pure emotional expressions are seldom elicited.
“First, ªpureº emotional expressions are seldom elicited.”
The paper relies on this without support.
Most of the time, people show blends of emotional expressions.
“Most of the time, people show blends of emotional expressions.”
The paper relies on this without support.
Some facial actions such as blink, wink, and lips sucked into the mouth are either encountered or not.
“Second, some facial actions such as blink, wink, and lips sucked into the mouth, are either encountered or not.”
The paper relies on this without support.
Each person has his or her own maximal intensity of displaying a particular facial action.
“In addition, each person has his/her own maximal intensity of displaying a particular facial action.”
The paper relies on this without support.
FACS provides an option to score intensity on a 3-level intensity scale for only five different AUs.
“First, there are merely five different AUs for which FACS provides an option to score intensity on a 3-level intensity scale (low, medium, and high).”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Automatic Analysis of Facial Expressions: The State of the Art > 6 CONCLUSION
“Also, all of the proposed approaches to automatic expression analysis perform only facial expression classification into the basic emotion categories defined by Ekman and Friesen [20].”
§ Automatic Analysis of Facial Expressions: The State of the Art > 5.2 Facial Expression Classification
“Four of the surveyed systems perform facial action coding from an input image or an image sequence (Table 7).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Automatic Analysis of Facial Expressions: The State of the Art > 2.3 Facial Expression Classification
“Ekman defined six such categories, referred to as the basic emotions: happiness, sadness, surprise, fear, anger, and disgust [19].”
§ Automatic Analysis of Facial Expressions: The State of the Art > References
“19. P. Eisert and B. Girod, ªAnalysing Facial Expressions for Virtual Conferencing,º IEEE Trans Computer Graphics and Applications, vol. 18, no. 5, pp. 70-78, 1998.”
Citations
Cites
Kass, M., Witkin, A., & Terzopoulos, D. (1988) · International Journal of Computer Vision · doi:10.1007/bf00133570 · Cited by 17,330
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1994) · Cited by 15,411
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
Numerical recipes in Pascal: the art of scientific computing
(1990) · Choice Reviews Online · doi:10.5860/choice.27-5158 · Cited by 12,040
(1981) · Cited by 11,963
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Horn, B. K. P., & Schunck, B. G. (1981) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.965761 · Cited by 7,443
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190
A direct adaptive method for faster backpropagation learning: the RPROP algorithm
Riedmiller, M., & Braun, H. (2002) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1993.298623 · Cited by 3,917
Neural network-based face detection
Rowley, H. A., Baluja, S., & Kanade, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655647 · Cited by 3,554
Research works printed but not linked: 13
- wiskott, 1995, labelled graphs and dynamic link matching for face recognition and scene analysis, Reihe Physik, 53
- cottrell, 1991, empath: fface, emotion, gender recognition using holons, Advances in Neural Information Processing Systems 3, 564
- vanger, 1995, applications of synergetics in decoding facial expression of eemotion, Proc Third Int l Conf Automatic Face and Gesture Recognition, 24
- izard, 1971, The Face of Emotion
- johanstone, 1995, acoustic profiles in prototypical vocal expressions of emotions, Proc Int l Conf Phonetic Science, 4, 2
- kawakami, 1995, 3d emotion space for interactive communication, Proc Computer Science Conf, 471
- ekman, 1978, Facial Action Coding System (FACS) Manual
- fridlund, 1987, facial expressions of emotion: review literature 1970-1983, Nonverbal Behavior and Communication, 143
- ellis, 1986, process underlying face recognition, The Neuropsychology of Face Perception and Facial Expression, 1
- bruce, 1986, Recognizing Faces
- matsuno, 1993, recognition of facial expression with potential net, Proc Asian Conf Computer Vision, 504
- zhao, 1996, classifying facial emotions by backpropagation neural networks with fuzzy inputs, Proc Conf Neural Information Processing, 1, 454
- picard, 1999, offline and online recognition of emotion expression from physiological data, Emotion-Based Agent Architectures Workshop Notes Int l Conf Autonomous Agents, 135
Cited by
Sorted by citations
Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003) · ACM Computing Surveys · doi:10.1145/954339.954342 · Cited by 6,174
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
“Much progress has been made in the last decade, and a thorough survey of the exiting work can be found in [ 1 ,2].”
BackgroundA Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Mitra, S., & Acharya, T. (2007) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.893280 · Cited by 2,048
“There are two major approaches to automated face recognition [43].”
BackgroundDeep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,175
“Exhaustive surveys on automatic expression analysis have been published in recent years [7], [8], [28], [29].”
BackgroundFace detection in color images
Hsu, R.-L., Abdel-Mottaleb, M. S., & Jain, A. K. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000242 · Cited by 1,884
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
Facial Expression Analysis under Partial Occlusion
Zhang, L., Verma, B., Tjondronegoro, D., & Chandran, V. (2018) · ACM Computing Surveys · doi:10.1145/3158369 · Cited by 1,338
“In earlier surveys on FEA [Pantic and Rothkrantz 2000], [Fasel and Luettin 2003], no study reported specifically designed techniques to overcome facial occlusion.”
BackgroundMethodInfluentialReliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
“expression information is either classified as a set of facial actions or a particular basic emotion [34].”
BackgroundA 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171
“The excellent review of recent advances in this field can be found in [27, 42, 46, 28, 15].”
BackgroundSocial signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842
“Face detection is typically the first step towards facial expression analysis [139] or gaze behavior analy-”
BackgroundEmotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806
Martínez, A. M. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1008382 · Cited by 823
Facial expression recognition from near-infrared videos
Zhao, G., Huang, X., Taini, M., Li, S. Z., & Pietikäinen, M. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.07.002 · Cited by 771
“In many cases, facial expression recognition research is based on static images [4,6,24,26] unlike in dynamic facial expression recognition, in which appearance andmotion information of the whole video sequence can be utilized as well.”
BackgroundScherer, K. R., Schuller, B., & Elkins, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 768
BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database
Zhang, X., Yin, L., Cohn, J. F., Canavan, S., Reale, M., Horowitz, A., Liu, P., & Girard, J. M. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.06.002 · Cited by 745
“, anger, disgust, fear, happiness, sadness, and surprise) [3,5,13].”
Kotsia, I., & Pitas, I. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.884954 · Cited by 687
“Facial expression recognition A survey on the research made regarding facial expression recognition can be found at [4], [5].”
BackgroundMultimodal human–computer interaction: A survey
Jaimes, A., & Sebe, N. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.019 · Cited by 965
Li, S., & Deng, W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 688
“AUTOMATIC facial expression recognition has made significant progress in the past two decades [56], [78].”
BackgroundFace recognition from a single image per person: A survey
Tan, X., Chen, S., Zhou, Z., & Zhang, F. (2006) · Pattern Recognition · doi:10.1016/j.patcog.2006.03.013 · Cited by 656
“All rights reserved. doi:10.1016/j.patcog.2006.03.013 [6], Pantic and Rothkrantz [7] surveyed the automatic facial expression analysis, Daugman [3] pointed out several critical issues involved in an effective face recognition system, while the most recent and comprehensive survey is possibly from that of Zhao et al. […”
BackgroundInfluentialRecognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.297 · Cited by 643
“The approach to automatic FACS coding presented here, in addition to being fully automated, also differs from approaches such as [ 13 ] and [16] in that instead of designing special purpose image features for each facial action, we explore general purpose learning mechanisms for data-driven facial expression classific…”
Facial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874
“Darwin [7] held an ethological view of emotional expressions , arguing that the expressions from infancy and lower life forms exist in adult humans.”
BackgroundAffective Body Expression Perception and Recognition: A Survey
Kleinsmith, A., & Bianchi‐Berthouze, N. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.16 · Cited by 624
Facial expression of pain: An evolutionary account
Williams, A. C. D. C. (2002) · Behavioral and Brain Sciences · doi:10.1017/s0140525x02000080 · Cited by 612
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Self-citationBartlett, M. S., Littlewort, G. C., Fasel, I., & Movellan, J. R. (2003) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2003.10057 · Cited by 580
Comparing Different Classifiers for Automatic Age Estimation
Lanitis, A., Draganova, C., & Christodoulou, C. C. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2003.817091 · Cited by 571
“In this context, systems capable of identifying faces [14], recognizing emotions [19], gender [17], and head orientation [16] have been developed.”
BackgroundAutomatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning
Ong, S. C. W., & Ranganath, S. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.112 · Cited by 562
“…is highly complex and understanding it involves a substantial commonality with research in machine analysis and understanding of human action and behavior; for example, face and facial expression recognition [84], [104], tracking and human motion analysis [53], [148], and gesture recognition [106].”
BackgroundCorneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
“tial surveys of these early works [214], [215], [216].”
BackgroundLocalizing Parts of Faces Using a Consensus of Exemplars
Belhumeur, P. N., Jacobs, D. W., Kriegman, D. J., & Kumar, N. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.23 · Cited by 536
“Facial expression recognition often involves detecting facial features and tracking them through a video sequence [22].”
BackgroundA Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition
Liu, Y., Zhang, J., Yan, W., Wang, S., Zhao, G., & Fu, X. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2485205 · Cited by 531
Local Directional Number Pattern for Face Analysis: Face and Expression Recognition
Rivera, A. R., Castillo, J. R., & Chae, O. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2235848 · Cited by 514
“Among these methods are Local Features Analysis [22], Gabor features [23], Elastic Bunch Graph Matching [24], and Local Binary Pattern (LBP) [14], [25].”
MethodFacial expression recognition via learning deep sparse autoencoders
Zeng, N., Zhang, H., Song, B., Liu, W., Li, Y., & Dobaie, A. M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.08.043 · Cited by 492
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks
Zhang, K., Huang, Y., Du, Y., & Wang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2689999 · Cited by 493
“focus on recognizing expressions from still frames [3].”
BackgroundSurvey on Emotional Body Gesture Recognition
Noroozi, F., Corneanu, C. A., Kamińska, D., Sapiński, T., Escalera, S., & Anbarjafari, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2874986 · Cited by 514
“phony [137], video conferencing and stress-monitoring tool, violence detection [138], [139], [140], video surveillance [141], and animation or synthesis of life-like agents [139] and auto-”
BackgroundActive and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 460
“A thorough survey of the existing works can be found in [12], [13], [14].”
BackgroundA survey on face detection in the wild: Past, present and future
Zafeiriou, S. P., Zhang, C., & Zhang, Z. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.03.015 · Cited by 476
“Automatic face detection is the cornerstone of all applications revolving around automatic facial image analysis including, but not limited to, face recognition and verification [1], face tracking for surveillance [2], facial behaviour analysis [3], facial attribute recognition [4] (i.”
BackgroundFacial Landmark Detection: A Literature Survey
Wu, Y., & Ji, Q. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1097-z · Cited by 443
“For example, facial expression recognition [68] and head pose estimation algorithms [66] may heavily rely on the facial shape information provided by the landmark locations.”
BackgroundHuman-Computer Interaction: Overview on State of the Art
Karray, F. O., Alemzadeh, M., Saleh, J. A., & Arab, M. N. (2008) · International Journal on Smart Sensing and Intelligent Systems · doi:10.21307/ijssis-2017-283 · Cited by 439
“Facial expression analysis generally deals with recognition of emotions visually [37] [38] [39].”
Facial Action Recognition for Facial Expression Analysis From Static Face Images
Pantić, M., & Rothkrantz, L. J. M. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.825931 · Cited by 425
Self-citationThe Human Face as a Dynamic Tool for Social Communication
Jack, R. E., & Schyns, P. G. (2015) · Current Biology · doi:10.1016/j.cub.2015.05.052 · Cited by 405
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580
“Few methods have been reported for automatic AU detection in face image sequences [16].”
MethodEmotion classification based on gamma-band EEG
Li, M., & Lu, B. (2009) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2009.5334139 · Cited by 413
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407
Habbal, A., Ali, M. K., & Abuzaraida, M. A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122442 · Cited by 407
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385
“A full survey of the field is out of scope for this paper, and we encourage the interested reader to review a number of excellent surveys for a more detailed description of the current state of the art [19], [11].”
BackgroundDynamics of facial expression extracted automatically from video
Littlewort, G. C., Bartlett, M. S., Fasel, I., Susskind, J. M., & Movellan, J. R. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 356
Automatic Analysis of Facial Actions: A Survey
Martínez, B., Valstar, M., Jiang, B., & Pantić, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2731763 · Cited by 370
“Existing works surveying methods on automatic facial expression recognition either focus on message-judgement approaches [61], [133], or contain just a limited subset of works on automatic AU detection [174], [214], or focus on”
BackgroundA high-resolution 3D dynamic facial expression database
Yin, L., Chen, X., Sun, Y., Worm, T., & Reale, M. (2008) · 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition · doi:10.1109/afgr.2008.4813324 · Cited by 488
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468
“There have been major advances in the computer vision literature for facial expression recognition over the past 15 years. See [22], [ 36 ] for reviews.”
BackgroundDeeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis
Liu, M., Li, S., Shan, S., Wang, R., & Chen, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16817-3_10 · Cited by 333
“Many works have been done in the literature [1,2], but it remains unsolved.”
BackgroundFacial Displays Are Tools for Social Influence
Crivelli, C., & Fridlund, A. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.02.006 · Cited by 355
“As early as 2000, it was recognized that the dominant approach of multivariate pattern classification (matching BETdesignated prototypic posed faces to their foreordained matching emotion labels) would be insufficient to capture: (i) the wide range of common facial displays that escape the prototypes, (ii) their all-i…”
BackgroundTelemental Health Care, an Effective Alternative to Conventional Mental Care: a Systematic Review
Langarizadeh, M., Tabatabaei, M., Tavakol, K., Naghipour, M., & Moghbeli, F. (2017) · Acta Informatica Medica · doi:10.5455/aim.2017.25.240-246 · Cited by 314
“We identified a large number of studies, reporting popular applications of telemental health care in various settings (1-8,11-28).”
BackgroundReal time facial expression recognition in video using support vector machines
Michel, P., & el Kaliouby, R. (2003) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/958432.958479 · Cited by 311
Toward Practical Smile Detection
Whitehill, J., Littlewort, G. C., Fasel, I., Bartlett, M. S., & Movellan, J. R. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.42 · Cited by 310
Meta-Analysis of the First Facial Expression Recognition Challenge
Valstar, M. F., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2200675 · Cited by 297
Ghimire, D., & Lee, J. (2013) · Sensors · doi:10.3390/s130607714 · Cited by 303
“The first survey on the research made regarding facial expression recognition was published in 1992 [4], and has been followed up by several researchers [5,6].”
BackgroundHamm, J., Kohler, C., Gur, R., & Verma, R. (2011) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2011.06.023 · Cited by 296
Stacked Hourglass Network for Robust Facial Landmark Localisation
Yang, J., Liu, Q., & Zhang, K. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.253 · Cited by 281
“facial attribute analysis [25], expression analysis [28], and face recognition [37, 46, 38, 21, 22, 34].”
BackgroundLearning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Liu, M., Shan, S., Wang, R., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.226 · Cited by 392
“Early research mostly focused on expression analysis from static facial images [20].”
BackgroundFace Expression Recognition and Analysis: The State of the Art
Bettadapura, V. (2012) · arXiv · doi:10.48550/arxiv.1203.6722 · Cited by 336
“Comprehensive and widely cited surveys by Pantic and Rothkrantz (2000) [7] and Fasel and Luttin (2003) [8] are available that perform an in-depth study of the published work from 1990 to 2001.”
BackgroundResultEmotion recognition using brain activity
Horlings, R., Datcu, D., & Rothkrantz, L. J. M. (2008) · International Conference on Computer Systems and Technologies (CompSysTech) · doi:10.1145/1500879.1500888 · Cited by 269
“For example, research has been done to make computers recognize emotion from speech [7], facial expressions [12] or a fusion of both methods [4].”
BackgroundSelf-citationEmotion Recognition through Multiple Modalities: Face, Body Gesture, Speech
Castellano, G., Kessous, L., & Caridakis, G. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85099-1_8 · Cited by 267
“Facial expressions [12] [13], vocal features [14] [15], body movements and postures [16] [17] [18], physiological signals [19] have been used as inputs during these attempts, while multimodal emotion recognition is currently gaining ground [20] [21] [22].”
BackgroundMethodTian, Y., Takeo Kanade, A., & Cohn, J. F. (2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1_19 · Cited by 268
“Two survey papers summarized the work (before year 1999) of facial expression analysis [35, 69].”
Lopatovska, I., & Arapakis, I. (2010) · Information Processing & Management · doi:10.1016/j.ipm.2010.09.001 · Cited by 279
Facial Expression Recognition Using Kernel Canonical Correlation Analysis (KCCA)
Zheng, W., Zhou, X., Zou, C., & Zhao, L. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.860849 · Cited by 263
Authentic facial expression analysis
Sebe, N., Lew, M. S., Sun, Y., Cohen, I., Gevers, T., & Huang, T. S. (2007) · Image and Vision Computing · doi:10.1016/j.imavis.2005.12.021 · Cited by 257
“The two recent surveys in the area [12, 22] provide an in depth review of many of the research done in recent years.”
BackgroundKessous, L., Castellano, G., & Caridakis, G. (2010) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0025-5 · Cited by 256
“A wide variety of machine learning techniques have been used in emotion recognition approaches [3, 21].”
BackgroundCohen, I., Cozman, F. G., Sebe, N., Cirelo, M. C., & Huang, T. S. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.127 · Cited by 255
“In [54], Pantic and Rothkrantz provide an in-depth review of much of the research done in automatic facial expression recognition in recent years.”
BackgroundMachine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248
“Since motion of the facial skin produces optical flow in the image, a large number of researchers have studied optical flow tracking (Pantic & Rothkrantz, 2000; 2003; Tian et al., 2005).”
InfluentialA high-resolution spontaneous 3D dynamic facial expression database
Zhang, X., Yin, L., Cohn, J. F., Canavan, S. J., Reale, M. J., Horowitz, A., & Liu, P. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553788 · Cited by 238
“, anger, disgust, fear, happiness, sadness, and surprise) [3][5][13].”
Automatic Facial Expression Recognition Using Facial Animation Parameters and Multistream HMMs
Aleksic, P. S., & Katsaggelos, A. K. (2006) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2005.863510 · Cited by 235
“Several automatic facial expression and facial action recognition systems [29]–[ 33 ] utilize FACS as facial features.”
MethodRecognition of facial expressions and measurement of levels of interest from video
Yeasin, M., Bullot, B., & Sharma, R. (2006) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.870737 · Cited by 234
“Such a system could also make classification of facial expressions widely accessible as a tool for research in behavioral science, computer games and medicine [9].”
MethodFusion of facial expressions and EEG for implicit affective tagging
Koelstra, S., & Patras, I. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.002 · Cited by 230
“Two main streams in the current research on automatic analysis of facial expressions consider facial affect (emotion) detection and facial muscle action (Action Unit) detection [18] [19] [20].”
BackgroundMeng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., & Wang, Y. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512532 · Cited by 225
“using static features extracted from still face images [31, 37].”
MethodTakeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226
An analysis of facial expression recognition under partial facial image occlusion
Kotsia, I., Buciu, I., & Pitas, I. (2007) · Image and Vision Computing · doi:10.1016/j.imavis.2007.11.004 · Cited by 223
“A survey on automatic facial expression recognition can be found in [6].”
BackgroundBody Movements for Affective Expression: A Survey of Automatic Recognition and Generation
Karg, M. E., Samadani, A.-A., Gorbet, R., Kühnlenz, K., Hoey, J., & Kulić, D. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.29 · Cited by 224
“Several surveys address detection of affective states in general [1], [2], [3], [4], [5], through facial or/and audio expressions [6], [7] and generation of affective expressions [4], [8].”
BackgroundMacro- and micro-expression spotting in long videos using spatio-temporal strain
Shreve, M., Godavarthy, S., Goldgof, D., & Sarkar, S. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771451 · Cited by 218
“Methods that do approach the problem of automatically spotting macro-expressions typically fall into three categories [10].”
BackgroundManifold based analysis of facial expression
Chang, Y. H., Hu, C., Feris, R. S., & Turk, M. (2005) · Image and Vision Computing · doi:10.1016/j.imavis.2005.08.006 · Cited by 214
“Some surveys [6,7] gave a detailed review of existing methods on facial expression analysis and recognition.”
BackgroundA video database of moving faces and people
O’Toole, A. J., Ha, J., Snow, S. L., Hurst, D. R., Pappas, M., Ayyad, J., & Abdi, H. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.90 · Cited by 213
“than those available previously, though see Pantic and Rohkrantz [24] for a review of the range of face images used in automatic analysis of facial expression.”
BackgroundMultimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network
Li, H., Sun, J., Xu, Z., & Chen, L. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2713408 · Cited by 211
“, [5], [6], [8], [18], [37], [49], [55], [56], [58], [61], [62]).”
BackgroundMethodInfluentialKrepki, R., Blankertz, B., Curio, G., & Müller, K. (2007) · Multimedia Tools and Applications · doi:10.1007/s11042-006-0094-3 · Cited by 213
A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models
Koelstra, S., Pantic, M., & Patras, I. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.50 · Cited by 242
Affective state estimation for human–robot Interaction
Kulic, D., & Croft, E. A. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.904899 · Cited by 230
“Recently, research has focused on using nonverbal communication, such as eye gaze [2], [3], facial expressions [11]–[13], and physiological signals [14]–[17] for human–robot and human–computer interaction.”
BackgroundJoint patch and multi-label learning for facial action unit detection
Zhao, K., Chu, W.-S., la Torre, F. D., Cohn, J. F., & Zhang, H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298833 · Cited by 205
“While significant progress has been made toward this goal [1,6,9,22], at least two critical problems remain.”
BackgroundPre-collision safety strategies for human-robot interaction
Kulić, D., & Croft, E. A. (2006) · Autonomous Robots · doi:10.1007/s10514-006-9009-4 · Cited by 200
“Recently, research has focused on using non-verbal communication, such as eye-gaze (Matsumoto et al., 1999, 2000), facial orientation (Stiefelhagen et al., 2001), facial expressions ( Pantic and Rothkrantz, 2000 ) and physiological signals (Sarkar, 2002; Rani et al., 2004 ;K uli´ and Croft, 2003; Picard, 2001 )f or hu…”
BackgroundLearning From Examples in the Small Sample Case: Face Expression Recognition
Guo, G., & Dyer, C. R. (2005) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2005.846658 · Cited by 196
“See [ 25 ] for a review of different approaches for FER.”
MethodDeep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013
Giannopoulos, P. G., Perikos, I., & Hatzilygeroudis, I. (2017) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-66790-4_1 · Cited by 200
Emotion recognition from unimodal to multimodal analysis: A review
Ezzameli, K., & Mahersia, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101847 · Cited by 206
An experimental comparison of gender classification methods
Mäkinen, E., & Raisamo, R. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.03.016 · Cited by 222
“Facial expression analysis has also received attention and there are two extensive surveys on it (Fasel and Luettin, 2003; Pantic and Rothkrantz, 2000).”
BackgroundFacial Expression Recognition in the Encrypted Domain Based on Local Fisher Discriminant Analysis
Rahulamathavan, Y., Phan, R. C. ‐., Chambers, J. A., & Parish, D. J. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.33 · Cited by 192
“As a result, facial expression recognition has become a prominent research topic in human-computer interaction, as well as in the fields of image processing, pattern recognition, machine learning, and human cognition [4], [5], [6].”
BackgroundBilinear Models for 3-D Face and Facial Expression Recognition
Mpiperis, I., Malassiotis, S., & Strintzis, M. G. (2008) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2008.924598 · Cited by 185
“Extensive surveys on 3D face recognition and facial expression recognition fr om 2D images and video may be found in [3] and [7] respectively.”
Kaulard, K., Cunningham, D. W., Bülthoff, H., & Wallraven, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0032321 · Cited by 195
Multimodal emotion recognition from expressive faces, body gestures and speech
Caridakis, G., Castellano, G., Kessous, L., Raouzaiou, A., Malatesta, L., Asteriadis, S., & Karpouzis, K. (2007) · IFIP The International Federation for Information Processing · doi:10.1007/978-0-387-74161-1_41 · Cited by 183
“Facial expressions [5], vocal features [6], body movements [7] have been used as inputs during these attempts, while multimodal emotion recognition is currently gaining ground [8].”
MethodTsiourti, C., Weiss, A., Wac, K., & Vincze, M. (2019) · International Journal of Social Robotics · doi:10.1007/s12369-019-00524-z · Cited by 182
“A modality is a sense, used to perceive signals from the outside world (i.e., sight, hearing) [9].”
BackgroundA survey of emotion recognition methods with emphasis on E-Learning environments
Imani, M., & Montazer, G. A. (2019) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102423 · Cited by 195
Multimodal assistive technologies for depression diagnosis and monitoring
Joshi, J., Goecke, R., Alghowinem, S., Dhall, A., Wagner, M., Epps, J., Parker, G. B., & Breakspear, M. J. (2013) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-013-0123-2 · Cited by 180
“Inferring emotions from facial expression analysis is a well-researched problem [31,41].”
BackgroundMachine learning methods for fully automatic recognition of facial expressions and facial actions
Bartlett, M. S., Littlewort, G. C., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398364 · Cited by 171
Metadata
- Authors
- Pantie, M., & Rothkrantz, L. J. M.
- Affiliations
- Delft University of Technology (NL)
- ORCID
- Rothkrantz, L.J.M.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2000
- Volume
- 22
- Issue
- 12
- Pages
- 1424-1445
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2159017231
- Semantic Scholar
- 087a6f471b5177014b08a342968901e1ef083ed1
- Cited by
- 1,778
- References
- 111
- Citations by year
- 2026: 33 · 2025: 31 · 2024: 30 · 2023: 43 · 2022: 41 · 2021: 43 · 2020: 58 · 2019: 65 · 2018: 72 · 2017: 71 · 2016: 86 · 2015: 87 · 2014: 105 · 2013: 105 · 2012: 108
- FWCI
- 23.88
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- Research Repository (Delft University of Technology) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression analysis, face detection, facial expression recognition, emotion classification, human visual system, human-machine interaction, real-time systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
