Modeling multimodal behaviors from speech prosody,
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the field of conversational agents, head gesture generation and evaluation has mainly focussed on head movements as co-speech speaker-in contrast to listener-behavior.
“In the field of conversational agents, head gesture generation and evaluation, has, however, mainly focussed on head movements (including nods) as co-speech speaker-in contrast to listenerbehavior (e.g., Lee and Marsella, 2010; Ding et al., 2013).”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Ding et al., 2013
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · see citing works
Hand and mind: what gestures reveal about thought
(1993) · Choice Reviews Online · doi:10.5860/choice.30-2492 · Cited by 5,090 · see citing works
Praat, a system for doing phonetics by computer
Boersma, P. (2002) · UvA-DARE (University of Amsterdam) · Cited by 4,417 · see citing works
Gesture Visible Action as Utterance
Kendon, A. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511807572 · Cited by 3,987 · see citing works
Hand and Mind: What Gestures Reveal about Thought
Feyereisen, P., & McNeill, D. M. (1994) · The American Journal of Psychology · doi:10.2307/1423299 · Cited by 1,713 · see citing works
Speech parameter generation algorithms for HMM-based speech synthesis
Tokuda, K., Yoshimura, T., Masuko, T., Kobayashi, T., & Kitamura, T. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2000.861820 · Cited by 1,004 · see citing works
Bolinger, D. L. (1989) · Stanford University Press eBooks · doi:10.1515/9781503623125 · Cited by 690 · see citing works
Cassell, J., Pélachaud, C., Badler, N. I., Steedman, M. J., Achorn, B., Becket, T., Douville, B., Prevost, S., & Stone, M. (1994) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/192161.192272 · Cited by 669 · see citing works
Parametric hidden Markov models for gesture recognition
Wilson, A. D., & Bobick, A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.790429 · Cited by 574 · see citing works
Visual Prosody and Speech Intelligibility: Head Movement Improves Auditory Speech Perception
Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502010.x · Cited by 482 · see citing works
Pakštas, A., Forchheimer, R., & Pandžić, I. S. (2002) · Wiley eBooks · doi:10.1002/0470854626 · Cited by 422 · see citing works
MPEG-4 Facial Animation: The Standard,Implementation and Applications
Pakštas, A., Forchheimer, R., & Pandzic, I. S. (2002) · John Wiley & Sons, Inc. eBooks · Cited by 346 · see citing works
Intonation and Its Uses: Melody in Grammar and Discourse
Bolinger, D. L. (1989) · Cited by 328 · see citing works
Generating Facial Expressions for Speech
Pélachaud, C., Badler, N. I., & Steedman, M. J. (1996) · Cognitive Science · doi:10.1207/s15516709cog2001_1 · Cited by 270 · see citing works
Rigid Head Motion in Expressive Speech Animation: Analysis and Synthesis
Busso, C., Deng, Z., Grimm, M., Neumann, U., & Narayanan, S. S. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.885910 · Cited by 181 · see citing works
A 3-D Audio-Visual Corpus of Affective Communication
Fanelli, G., Gall, J., Romsdorfer, H., Weise, T., & Gool, L. V. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2052239 · Cited by 162 · see citing works
Mixed excitation for HMM-based speech synthesis
Yoshimura, T., Tokuda, K., Masuko, T., Kobayashi, T., & Kitamura, T. (2001) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2001-539 · Cited by 157 · see citing works
Levine, S., Krähenbühl, P., Thrun, S., & Koltun, V. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778861 · Cited by 139 · see citing works
Rule-based visual speech synthesis
Beskow, J. (1995) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1995-81 · Cited by 113 · see citing works
Natural head motion synthesis driven by acoustic prosodic features
Busso, C., Deng, Z., Neumann, U., & Narayanan, S. S. (2005) · Computer Animation and Virtual Worlds · doi:10.1002/cav.80 · Cited by 86 · see citing works
How to Train Your Avatar: A Data Driven Approach to Gesture Generation
Chiu, C., & Marsella, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23974-8_14 · Cited by 74 · see citing works
Generating Human-Like Behaviors Using Joint, Speech-Driven Models for Conversational Agents
Mariooryad, S., & Busso, C. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2012.2201476 · Cited by 58 · see citing works
Human ethology : claims and limits of a new discipline : contributions to the Colloquium
von Cranach, M., & Forschung, W.-R.-S. F. A. (1979) · Cambridge University Press eBooks · Cited by 45 · see citing works
Predicting Speaker Head Nods and the Effects of Affective Information
Lee, J., & Marsella, S. C. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2051874 · Cited by 37 · see citing works
Bevacqua, E., Prépin, K., Niewiadomski, R., de Sevin, É., & Pélachaud, C. (2010) · Natural language processing · doi:10.1075/nlp.8.20bev · Cited by 31 · see citing works
Modeling Speaker Behavior: A Comparison of Two Approaches
Lee, J., & Marsella, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33197-8_17 · Cited by 20 · see citing works
Speech driven head motion synthesis based on a trajectory model
Hofer, G., Shimodaira, H., & Yamagishi, J. (2007) · ACM SIGGRAPH Posters · doi:10.1145/1280720.1280814 · Cited by 17 · see citing works
Contextual Hidden Markov Models
Radenen, M., & Artières, T. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288328 · Cited by 13 · see citing works
Self-citationSpeech-driven eyebrow motion synthesis with contextual Markovian models
Ding, Y., Radenen, M., Artières, T., & Pélachaud, C. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638360 · Cited by 13 · see citing works
Self-citationAcoustically-Driven Talking Face Synthesis using Dynamic Bayesian Networks
Xue, J., Borgstrom, J., Jiang, J., Bernstein, L. E., & Alwan, A. A. (2006) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2006.262743 · Cited by 12 · see citing works
HMM-based automatic eye-blink synthesis from speech
Dziemianko, M., Hofer, G., & Shimodaira, H. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-153 · Cited by 5 · see citing works
Levine, S., Krähenbühl, P., Thrun, S., & Koltun, V. (2010) · ACM Transactions on Graphics · doi:10.1145/1833351.1778861 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset
Zhang, Z., Li, L., Ding, Y., & Fan, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00366 · Cited by 368 · see citing works
Self-citationAudio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion
Wang, S., Li, L., Ding, Y., Fan, C., & Yu, X. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/152 · Cited by 169 · see citing works
Self-citationDing, Y., Prépin, K., Huang, J., Pélachaud, C., & Artières, T. (2014) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/copn4977 · Cited by 38 · see citing works
Self-citationModeling Feedback in Interaction With Conversational Agents—A Review
Axelsson, A., Buschmeier, H., & Skantze, G. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.744574 · Cited by 33 · see citing works
“In the field of conversational agents, head gesture generation and evaluation, has, however, mainly focussed on head movements (including nods) as co-speech speaker—in contrast to listener— behavior (e.g., Lee and Marsella, 2010; Ding et al., 2013).”
BackgroundAudio-Driven Robot Upper-Body Motion Synthesis
Ondras, J., Çeliktutan, O., Bremner, P., & Güneş, H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2966730 · Cited by 29 · see citing works
Multimodal Behavior Modeling for Socially Interactive Agents
Pélachaud, C., Busso, C., & Heylen, D. (2021) · ACM eBooks · doi:10.1145/3477322.3477331 · Cited by 24 · see citing works
Meaningful head movements driven by emotional synthetic speech
Sadoughi, N., Liu, Y., & Busso, C. (2017) · Speech Communication · doi:10.1016/j.specom.2017.07.004 · Cited by 23 · see citing works
Joint Learning of Speech-Driven Facial Motion with Bidirectional Long-Short Term Memory
Sadoughi, N., & Busso, C. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67401-8_49 · Cited by 20 · see citing works
A Multifaceted Study on Eye Contact based Speaker Identification in Three-party Conversations
Ding, Y., Zhang, Y., Xiao, M., & Deng, Z. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025644 · Cited by 20 · see citing works
Self-citationA Music-Driven Deep Generative Adversarial Model for Guzheng Playing Animation
Chen, J., Fan, C., Zhang, Z., Li, G., Zhao, Z., Deng, Z., & Ding, Y. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3115902 · Cited by 19 · see citing works
Self-citationSuggestions for Extending SAIBA with the VIB Platform
Pécune, F., Cafaro, A., Chollet, M., Philippe, P., & Pélachaud, C. (2019) · HAL · doi:10.2390/biecoll-wasiva2014-03 · Cited by 16 · see citing works
Towards Multimodal Human-Like Characteristics and Expressive Visual Prosody in Virtual Agents
Fares, M. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3421155 · Cited by 15 · see citing works
Statistics‐based Motion Synthesis for Social Conversations
Yang, Y., Yang, J., & Hodgins, J. K. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14114 · Cited by 13 · see citing works
Part-of-Speech and Prosody-based Approaches for Robot Speech and Gesture Synchronization
Pérez-Mayos, L., Farrús, M., & Adell, J. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-019-01100-3 · Cited by 12 · see citing works
Transformer Network for Semantically-Aware and Speech-Driven Upper-Face Generation
Fares, M., Pélachaud, C., & Obin, N. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909519 · Cited by 10 · see citing works
Real-Time Visual Prosody for Interactive Virtual Agents
van Welbergen, H., Ding, Y., Sattler, K., Pélachaud, C., & Kopp, S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-21996-7_16 · Cited by 10 · see citing works
Self-citationLow-Level Characterization of Expressive Head Motion Through Frequency Domain Analysis
Ding, Y., Shi, L., & Deng, Z. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2805892 · Cited by 10 · see citing works
Self-citationAn embodied virtual agent platform for emotional Stroop effect experiments: A proof of concept
Oker, A., Glas, N., Pécune, F., & Pélachaud, C. (2018) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2018.04.011 · Cited by 8 · see citing works
Audio-Driven Laughter Behavior Controller
Ding, Y., Huang, J., & Pélachaud, C. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2754365 · Cited by 7 · see citing works
Self-citationPerceptual enhancement of emotional mocap head motion: An experimental study
Ding, Y., Shi, L., & Deng, Z. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273607 · Cited by 7 · see citing works
Self-citationVers des Agents Conversationnels Animés dotés d'émotions et d'attitudes sociales
Ochs, M., Ding, Y., Fourati, N., Chollet, M., Ravenet, B., Pécune, F., Glas, N., Prépin, K., Clavel, C., & Pélachaud, C. (2015) · Journal d Interaction Personne-Système · doi:10.46298/jips.1282 · Cited by 5 · see citing works
Self-citationData-driven Communicative Behaviour Generation: A Survey
Oralbayeva, N., Aly, A., Sandygulova, A., & Belpaeme, T. (2023) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3609235 · Cited by 5 · see citing works
Transformer Network for Semantically-Aware and Speech-Driven Upper-Face GenerationPreprint
Fares, M., Pélachaud, C., & Obin, N. (2021) · HAL · doi:10.48550/arxiv.2110.04527 · Cited by 5 · see citing works
Vers des Agents Conversationnels Animés Socio-Affectifs
Ochs, M., Ding, Y., Fourati, N., Chollet, M., Ravenet, B., Pécune, F., Glas, N., Prépin, K., Clavel, C., & Pélachaud, C. (2013) · Conference on l'Interaction Homme-Machine (IHM) · doi:10.1145/2534903.2534913 · Cited by 3 · see citing works
Self-citationNguyen, D.-C., Bailly, G., & Elisei, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-91250-9_13 · Cited by 3 · see citing works
Learning Activity Patterns Performed With Emotion
Wang, Q., Artières, T., & Ding, Y. (2016) · International Conference on Movement and Computing (MOCO) · doi:10.1145/2948910.2948958 · Cited by 2 · see citing works
Self-citationFotedar, G., & Ghosh, P. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-999 · Cited by 2 · see citing works
MỘT KỸ THUẬT MÔ PHỎNG CỬ ĐỘNG CỦA MÔ HÌNH ĐẦU NGƯỜI 3D THEO LỜI THOẠI TIẾNG VIỆT
Phương, T. T., Vinh, N. D., Toàn, H. M., Toàn, Đ. N., & Ngọc, N. M. (2023) · TNU Journal of Science and Technology · doi:10.34238/tnu-jst.8297 · Cited by 1 · see citing works
I-Brow: Hierarchical and Multimodal Transformer Model for Eyebrows Animation Synthesis
Fares, M., Pélachaud, C., & Obin, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35894-4_33
Multimodal Expressive Gesturing With Style
Fares, M. (2023) · theses.fr (ABES) · doi:10.70675/a1375d8dzd155z4b3ezb79azc121e1ceecad
Modelling communicative behaviours for different roles of pedagogical agents
Krishna, S. (2021) · theses.fr (ABES) · doi:10.70675/f8e9d4b6z9585z4883z8fa1z2afa484bd27b
- Authors
- Ding, Y., Pelachaud, C., & Artières, T.
- Affiliations
- Télécom Paris (FR)
- LIP6 (FR)
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Berlin Heidelberg
- Published
- 2013
- Volume
- 8108
- Pages
- 217-228
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W56717636
- Cited by
- 32
- References
- 32
- Citations by year
- 2023: 4 · 2022: 2 · 2021: 6 · 2020: 3 · 2019: 2 · 2018: 4 · 2017: 6 · 2016: 1 · 2015: 2 · 2014: 1 · 2013: 1
- FWCI
- 2.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- HAL (submittedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- prosody, facial animation, virtual agents, nonverbal communication, multimodal behavior, hidden Markov models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
