A Survey of Listener Behavior and Listener Models for Embodied Conversational Agents
Abstract
The usage of multimodal corpora is crucial for studying human behavior as well as modeling embodied conversational agents (Cowie et al., 2010a). Gathering data and creating corpora is complex. Interested readers can read work by Cowie and Douglas-Cowie on the topic (Cowie et al., 2010b) wherein different methodologies for creating corpora are described.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There is a lack of reviews providing a more holistic overview of feedback in human-agent conversation that do not just focus on backchannels and that take both human-agent and agent-human feedback into account.
“However, we have noticed a lack of reviews providing a more holistic overview of the area, which do not just focus on backchannels, and which take both human-agent and agent-human feedback into account.”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Bevacqua, 2013
Bevacqua (2009) found that positive and negative feedback can be reliably separated but that signals can be both polysemous and synonymous regarding more specific functions.
“For the model it was studied (details in Bevacqua et al., 2007; Bevacqua, 2013) which meaning humans assign to these signals, finding that positive and negative feedback can be reliably separated but that signals can be both polysemous and synonymous regarding more specific functions-i.e., users may assign different meanings to the same signal and the same meaning to multiple signals.”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Bevacqua, 2013
A systematic study of different nonverbal feedback behaviors does not exist yet.
“A systematic study of different nonverbal feedback behaviors does not exist, yet (but see Bevacqua, 2013 for a survey).”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Bevacqua, 2013
Cites
Gumperz, J. J. (1982) · doi:10.1017/cbo9780511611834 · Cited by 5,861 · see citing works
The chameleon effect: the perception–behavior link and social interaction
Chartrand, T. L., & Bargh, J. A. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.76.6.893 · Cited by 3,674 · see citing works
The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding
EKMAN, P., & FRIESEN, W. V. (1969) · Semiotica · doi:10.1515/semi.1969.1.1.49 · Cited by 3,316 · see citing works
Some functions of gaze-direction in social interaction
Kendon, A. (1967) · Acta Psychologica · doi:10.1016/0001-6918(67)90005-4 · Cited by 2,524 · see citing works
Clark, H. H., & Schaefer, E. F. (1989) · Cognitive Science · doi:10.1207/s15516709cog1302_7 · Cited by 1,699 · see citing works
Some signals and rules for taking speaking turns in conversations.
Duncan, S. (1972) · Journal of Personality and Social Psychology · doi:10.1037/h0033031 · Cited by 1,386 · see citing works
Cybernetics, or control and communication in the animal and the machine (2nd ed.).
Wiener, N. (1961) · doi:10.1037/13140-000 · Cited by 1,068 · see citing works
On the semantics and pragmatics of linguistic feedback
ALLWOOD, J., NIVRE, J., & AHLSÉN, E. (1992) · Journal of Semantics · doi:10.1093/jos/9.1.1 · Cited by 442 · see citing works
Prosodic features which cue back-channel responses in English and Japanese
Ward, N., & Tsukahara, W. (2000) · Journal of Pragmatics · doi:10.1016/s0378-2166(99)00109-5 · Cited by 432 · see citing works
Creating Rapport with Virtual Agents
Gratch, J., Wang, N., Gerten, J., Fast, E., & Duffy, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74997-4_12 · Cited by 419 · see citing works
The power of a nod and a glance: envelope vs. emotional feedback in animated conversational agents
Cassell, J., & Thorisson, K. R. (1999) · Applied Artificial Intelligence · doi:10.1080/088395199117360 · Cited by 383 · see citing works
Between Speaking and Listening: The Vocalisation of Understandings1
Gardner, R. (1998) · Applied Linguistics · doi:10.1093/applin/19.2.204 · Cited by 286 · see citing works
Chartrand, T. L., Maddux, W. W., & Lakin, J. L. (2006) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195307696.003.0014 · Cited by 268 · see citing works
Discourse‐oriented facial displays in conversation
Chovil, N. (1991) · Research on Language and Social Interaction · doi:10.1080/08351819109389361 · Cited by 255 · see citing works
Non-verbal cues for discourse structure
Cassell, J., Nakano, Y., Bickmore, T., Sidner, C. L., & Rich, C. (2001) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1073012.1073028 · Cited by 183 · see citing works
Brunner, L. J. (1979) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.37.5.728 · Cited by 154 · see citing works
Head movement during listening turns in conversation
Hadar, U., Steiner, T. J., & Rose, F. C. (1985) · Journal of Nonverbal Behavior · doi:10.1007/bf00986881 · Cited by 151 · see citing works
Natural Behavior of a Listening Agent
Maatman, R. M., Gratch, J. S., & Marsella, S. (2005) · Lecture notes in computer science · doi:10.1007/11550617_3 · Cited by 129 · see citing works
Multimodal expressive embodied conversational agents
Pélachaud, C. (2005) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1101149.1101301 · Cited by 126 · see citing works
Predicting Listener Backchannels: A Probabilistic Multimodal Approach
Morency, L., de Kok, I., & Gratch, J. S. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85483-8_18 · Cited by 109 · see citing works
Þórisson, K. R. (1997) · doi:10.1145/267658.267823 · Cited by 75 · see citing works
Using prosodic clues to decide when to produce back-channel utterances
Ward, N. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.607961 · Cited by 57 · see citing works
Multimodal Backchannels for Embodied Conversational Agents
Bevacqua, E., Pammi, S., Hyniewska, S., Schröder, M., & Pélachaud, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15892-6_21 · Cited by 56 · see citing works
Self-citationSearching for prototypical facial feedback signals,
Heylen, D., Bevacqua, E., Tellier, M., & Pelachaud, C. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74997-4_14 · Cited by 41 · see citing works
Self-citationMultimodal feedback cues in human-machine interactions
Granström, B., House, D., & Swerts, M. G. J. (2002) · Speech prosody · doi:10.21437/speechprosody.2002-72 · Cited by 37 · see citing works
A listener model: introducing personality traits
Bevacqua, E., de Sevin, É., Hyniewska, S., & Pélachaud, C. (2012) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-012-0094-8 · Cited by 31 · see citing works
Self-citationAffective Feedback in a Tutoring System for Procedural Tasks
Heylen, D., Vissers, M., den Akker, R. O., & Nijholt, A. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24842-2_26 · Cited by 28 · see citing works
THE SYNTAXES OF BODILY COMMUNICATION
Argyle, M. (1973) · Linguistics · doi:10.1515/ling.1973.11.112.71 · Cited by 26 · see citing works
Sidner, C. L., Kidd, C. D., Lee, C. W., & Lesh, N. B. (2004) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/964456.964458 · Cited by 23 · see citing works
40. Mind, hands, face, and body: A sketch of a goal and belief view of multimodal communication
Poggi, I. (2013) · De Gruyter eBooks · doi:10.1515/9783110261318.627 · Cited by 14 · see citing works
Cross-language analysis of voice onset time in stuttered speech
Rezaei-Aghbash, N., Whiteside, S. P., & Cudd, P. A. (1999) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1999-272 · Cited by 12 · see citing works
Heylen, D. (2005) · University of Twente Research Information · Cited by 10 · see citing works
Multimodal Backchannel Generation for Conversational Agents
Heylen, D. (2007) · University of Twente Research Information · Cited by 9 · see citing works
Conceptual discrepancies and feedback in human-computer interaction
Beun, R., & van Eijk, R. M. (2004) · doi:10.1145/1005220.1005237 · Cited by 6 · see citing works
Speech Synthesis and Discourse Information
Campbell, N. (2006) · doi:10.5281/zenodo.14167413 · Cited by 1 · see citing works
Flagged first, then by citations
1 of the 36 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Ruhland, K., Peters, C., Andrist, S., Badler, J. B., Badler, N. I., Gleicher, M. L., Mutlu, B., & McDonnell, R. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12603 · Cited by 218 · see citing works
Look me in the Eyes: A Survey of Eye and Gaze Animation for Virtual Agents and Artificial Systems
Ruhland, K., Andrist, S., Badler, J. B., Peters, C., Badler, N. I., Gleicher, M. L., Mutlu, B., & McDonnell, R. (2014) · HAL · doi:10.2312/egst.20141036 · Cited by 70 · see citing works
Modeling Feedback in Interaction With Conversational Agents—A Review
Axelsson, A., Buschmeier, H., & Skantze, G. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.744574 · Cited by 33 · see citing works
“There already exist a few review articles that focus on how agents should be able to produce feedback in the form of backchannels (brief listener responses; de Kok and Heylen, 2012; Bevacqua, 2013).”
BackgroundMethodAutomatic and Manual Annotations in First Encounter Dialogues
Jokinen, K., & Wilcock, G. (2014) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-442-8-175 · Cited by 2 · see citing works
- Authors
- Bevacqua, E.
- Corresponding author
- Bevacqua, Elisabetta
- ORCID
- Bevacqua, Elisabetta
- Venue
- Coverbal Synchrony in Human-Machine Interaction
- Publisher
- CRC Press
- Published
- 2013-10-16
- Pages
- 243-268
- OpenAlex
- W2503663930
- Cited by
- 4
- References
- 36
- Citations by year
- 2022: 1 · 2015: 1 · 2014: 2
- FWCI
- 2.22
- Type
- book chapter
- Language
- en
- Open access
- closed
- Locations
- CRC Press eBooks (publishedVersion)
- HAL (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Arts and Humanities › Language and Linguistics › Language, Discourse, Communication Strategies
- Keywords
- listener behavior, embodied conversational agents, multimodal corpus, corpus construction, human-agent interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
