The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Being bored? Recognising natural interest by extensive audiovisual integration for real-life application
Open accessAbstract
Automatic detection of the level of human interest is of high relevance for many technical applications, such as automatic customer care or tutoring systems. However, the recognition of spontaneous interest in natural conversations independently of the subject remains a challenge. Identification of human affective states relying on single modalities only is often impossible, even for humans, since different modalities contain partially disjunctive cues. Multimodal approaches to human affect recognition generally are shown to boost recognition performance, yet are evaluated in restrictive laboratory settings only. Herein we introduce a fully automatic processing combination of Active-Appearance-Model-based facial expression, vision-based eye-activity estimation, acoustic features, linguistic analysis, non-linguistic vocalisations, and temporal context information in an early feature fusion process. We provide detailed subject-independent results for classification and regression of the Level of Interest using Support-Vector Machines on an audiovisual interest corpus ( AVIC ) consisting of spontaneous, conversational speech demonstrating "theoretical" effectiveness of the approach. Further, to evaluate the approach with regards to real-life usability a user-study is conducted for proof of "practical" effectiveness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · cited 1×
“These weak classifiers are combined by a Gentle AdaBoost [46], which tries to reject windows without a face at early stages of a cascade.”
MethodA Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Assessing Agreement on Classification Tasks: The Kappa Statistic
Carletta, J. (1996) · arXiv · doi:10.48550/arxiv.cmp-lg/9602004 · Cited by 2,088 · cited 1×
“Detailed inter-annotator kappa-values were computed according to [27] and are given in Table 5.”
MethodAutomatic recognition and analysis of human faces and facial expressions: a survey
Samal, A., & Iyengar, P. A. (1992) · Pattern Recognition · doi:10.1016/0031-3203(92)90007-6 · Cited by 910 · cited 1×
“…number of approaches towards recognition of such states [31], reaching from computationally less demanding low-level features such as global motions [32] over extraction of MPEG7 facial feature points [33] to high-level action units, including such in a 3D space (cf. [34,35,31] for overviews).”
BackgroundFacial and vocal expressions of emotion
Russell, J. A., Bachorowski, J.-A., & Fernández-Dols, J.-M. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145102 · Cited by 873
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
The painful face – Pain expression recognition using active appearance models
Ashraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K. M., & Solomon, P. E. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.05.007 · Cited by 355
Machine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248 · cited 2×
“…number of approaches towards recognition of such states [31], reaching from computationally less demanding low-level features such as global motions [32] over extraction of MPEG7 facial feature points [33] to high-level action units, including such in a 3D space (cf. [34,35,31] for overviews).”
BackgroundAffect Recognition from Face and Body: Early Fusion vs. Late Fusion
Gunes, H., & Piccardi, M. (2006) · 2005 IEEE International Conference on Systems, Man and Cybernetics · doi:10.1109/icsmc.2005.1571679 · Cited by 191
Cited by
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818
Self-citationSchuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Self-citationAkçay, M. B., & Oğuz, K. (2020) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 721
The INTERSPEECH 2010 paralinguistic challenge
Schuller, B., Steidl, S., Batliner, A., Burkhardt, F., Devillers, L., Müller, C., & Narayanan, S. S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-739 · Cited by 571
Self-citationFeatures and classifiers for emotion recognition from speech: a survey from 2000 to 2011
Anagnostopoulos, C.-N., Iliou, T., & Giannoukos, I. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9368-5 · Cited by 428
OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit
Eyben, F., Wollmer, M., & Schuller, B. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349350 · Cited by 418
“Finally, four ready-to-use model-sets are provided for recognition of six basic emotion categories (trained on the Berlin Speech Emotion Database (EMO-DB) [2] and the eNTERFACE database), for recognition of emotion in a continuous three-dimensional feature space spanned by activation, valence, and dominance (trained o…”
MethodInfluentialSelf-citationCross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B., Vlasenko, B., Eyben, F., Wollmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 388
Self-citationSparse Autoencoder-Based Feature Transfer Learning for Speech Emotion Recognition
Deng, J., Zhang, Z., Marchi, E., & Schuller, B. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.90 · Cited by 366
Self-citationGunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360
Self-citationLotfian, R., & Busso, C. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2736999 · Cited by 329
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288
Self-citationDeep neural networks for acoustic emotion recognition: Raising the benchmarks
Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., & Schuller, B. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947651 · Cited by 288
“In the case of 10 or fewer speakers in one corpus we apply the LOSO strategy; otherwise, namely for AVIC, eNTERFACE, SmartKom, and VAM, we select 5 speaker groups with utmost equal number of male and female speakers and samples per group for LOSGO evaluation.”
BackgroundSelf-citationParalinguistics in speech and language—State-of-the-art and the challenge
Schuller, B., Steidl, S., Batliner, A., Burkhardt, F., Devillers, L., Müller, C., & Narayanan, S. (2013) · Computer Speech & Language · doi:10.1016/j.csl.2012.02.005 · Cited by 284
Self-citationAcoustic emotion recognition: A benchmark comparison of performances
Schuller, B., Vlasenko, B., Eyben, F., Rigoll, G., & Wendemuth, A. (2009) · 2009 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2009.5372886 · Cited by 267
Self-citationBuilding autonomous sensitive artificial listeners
Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., ter Maat, M., McKeown, G., Pammi, S., Pantic, M., et al. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.34 · Cited by 185
Self-citationZhang, Y., Liu, Y., Weninger, F., & Schuller, B. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953106 · Cited by 55
“The Audiovisual Interest Corpus (AVIC) [24] features the level of interest of subjects in the scenario in which a product presenter led them through a commercial advertisement.”
Self-citationTandem decoding of children's speech for keyword detection in a child-robot interaction scenario
Wöllmer, M., Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · ACM Transactions on Speech and Language Processing · doi:10.1145/1998384.1998386 · Cited by 18
Self-citationChiba, Y., Nose, T., & Ito, A. (2017) · 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8282069 · Cited by 11
Metadata
- Authors
- Schuller, B., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Höthker, A., & Konosu, H.
- Venue
- Image and Vision Computing
- Publisher
- Elsevier BV
- Published
- 2009-11
- Volume
- 27
- Issue
- 12
- Pages
- 1760-1774
- ISSN
- 0262-8856 (ISSN-L)
- OpenAlex
- W2032382719
- Cited by
- 194
- References
- 12
- FWCI
- 28.12
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.